I agree with both happiness and confidence, and also I'd say having a partner who respects you and treats you well. When someone is always picking on your appearance it's hard to feel in the mood. Someone who isn't interested in your pleasure can really mess things up too.
Although, I guess being truly confident in the first place will make you walk away from people like that quickly, as you should.