Anal sex should absolutely never hurt. If it hurts, something isn't right. You should always start small, finger sized or smaller, and use lots and lots of lube, then a little extra lube on top of that. Then work up slowly, until you can take whatever you're trying to take without pain.
This reminds me of that exchange from the first episode of queer as folk:
Justin: It hurts. Does it always hurt?
Brian: Alittle. But that's part of it.
But they were talking about going from zero to sixty- from never having been anally penetrated to full on buttfucking with a sizable cock. Of course that's gonna hurt.
Anal sex should not be inherently painful. If you warm up first (rimming, rubbing with lubed finders, just stimulation without penetration) and use smaller stuff to start with (fingers, small plugs), it's unlikely that you will experience any pain at all. If you start off this way, your body will learn to associate anal play with pleasure instead of pain. It will make moving up to larger stuff (like cocks) easier in the long run. Trying to take something really large (like a fist) can cause some pain, which works for some folks. But anal sex should not be painful when you're just starting out.
Because you have mentioned pain right off the bat and expressed curiousity about toys, it sounds like you have skipped the "use smaller stuff" stage. Toys are absolutely a great option for newbies and experienced buttfuckers alike, and they come in all sizes. I'd suggest backing off whatever is causing you pain and go for something smaller.