I'll admit I've often had the feeling that it's just not fair, when he's too warm he can strip his upper half right down to skin, but I have to at least have a bikini top on. When I'm too warm I don't want anything on my skin at all.
That being said, I can see why people feel women should keep them covered. A woman's chest is much more intimate and sexual than a man's. His breasts don't feed his young and they never will, they probably don't give him nearly as much sexual pleasure (though some men's nipples are more sensitive than others), and they are definitely not looked at as something a cock could go in between.
So too do I feel an intense sense of embarrassment or even downright shame if my cleavage is too deep, or if I raise my arms and my shirt lifts to expose the underside of my breasts. Sadly I feel that until the sexual weight is removed from women's breasts, we should probably keep them covered, at least in public scenarios.