It can be annoying, if you don't do it correctly. What I found helps is to keep the contest running for no longer than a few weeks, so people don't forget about it and so you don't get completely overwhelmed. Be very specific about how people enter- do they comment on your post? Email you? Tweet about it? etc.
I also found it helpful to keep a word document where everyone who entered was added to a numbered list, as they entered. That way I was keeping track of things as they were happening, instead of leaving it until the last minute. I always chose my winners via a random number generator as well, so that way you can just referred to your list of entrants to see which name matches up with that number.