I think men (in the US anways), tend to associate anything to do with their ass as being gay if it is being played with. There is a fear, as illogical as one can be, that by enjoying fun through the ass the act will make one gay by extension. I discovered my Pspot when I was 29, and before that time there was no way I would have considered putting something back there, nor would I have been too keen to have a girl put her finger there. Now of course that is a completely different story.
As Scott rightly points out, this of course cuts both ways. Most women, in my experience, have their own issues with trying anything anal. Interestingly, I have found that those who are the most vehemently opposed to assplay seem to get off the most when assplay is properly introduced at some point. I think for women (and this is jut me postulating), there are a couple of different things going on. On one level I think women associate anal sex with being a slut or a dirty girl. Society says this is bad, just as society has long that being gay is bad, so therefore anything anal is bad for men too. Perhaps more important is the power of the spoken word. More often than not, women have an experience with anal that involves their boyfriend or guy they are having sex with jamming their dick in their ass. Some people like this, but most really need some teasing in the first place to make things pleasurable. In any event, horror stories about the time girls had painful anal sex dominate the discussion, and, as a result, you have a pretty effective impediment to women wanting anything to do with anal through these tales.