I think a lot of people live in very small worlds, at least with regard to sex and gender.
They don't think about these things, they believe the current "system" benefits them or at least they don't see it causing themselves harm.
They don't think about gender and sex as anything more than male or female. They don't think about their own gender expression, or even their own sexuality all that much. They take them for granted and never learn or explore.
And in America, at least, I feel that certain religious ideas do have an impact. I don't believe all religious people are transphobic or bad ! It's just an attitude that comes with a certain kind of faith that says it is your personal job to make sure people are on the right path. I know religious people who are NOT like that, but unfortunately some are and are vocal about it, telling people that they must live a certain way because their interpretation of scripture or personal norms dictate it (and certainly, non-religious people can be guilty of this as well).