Nope. Some things I don't eat and I can still be incredibly polite. I simply say "no thank you", or "could I please have a double helping of ....whatever, instead". Only once, when someone "INSISTED" I taste their special dish, did I take them aside, and say "oh, dear, your dish smells positively heavenly, but I have to tell you, I am allergic to that food (a tiny white lie), and she said "OH, I so understand", and that was that. In fact, I wrote her a NICE thank you note for the wonderful meal that evening and I have been invited back to EVERY party she has (she has LOTS), and so I KNOW she was not insulted. In fact, never has she served that specific dish, or at least without an alternative, at any party I have been to at her place. You can hold to your limits and still be VERY kind and VERY polite.