If you don't want to do it then don't. Talk with your husband about it and find something else to do - he should love and cherish you enough to want you to be comfortable.
That said do figure out why, both for you as a couple and you personally. Sometimes you can get around things when you think about them - for instance, if it's a cleanliness issue then use a barrier (slit a glove or use plastic wrap - crumple it up a few times to keep it from being too clingy). There's no reason that you should feel you have to do something you don't want to, so if you don't get around it then don't do it.