Well, if I have to pick one of the two I'd go with contacts. But I don't really wear my glasses when I'm in bed unless I'm reading, so even if I were to have sex with my glasses on I'd try to find a way to remove them without breaking focus or my glasses.
I wear glasses in my day to day life when necessary (I can't see things that are far away), but take them off for sex - they get in the way too much. I'll wear them sometimes as part of a particular look my man likes (naughty librarian) but they have to come off eventually.
Usually I will wear my contacts, because that's what I wear every day anyway. I usually only wear my glasses right before I go to bed. If I was already wearing my glasses and we were going to have sex I would take them off. Every partner I've had who wears glasses would take them off before sex also.