Really? "Most of us?" Why is that ever funny? And why do you presume that others think it's funny?
A lot of bigoted pricks think it's hilarious to make jokes about whether a pal is black because maybe they love watermelon or have a dark tan. Basically, you straights are making the same kind of bigoted "joke" at the expense of queer men. I think it would be a good idea to examine that statement... and your sense of humor, too. Just sayin'.