I have used one of the biggest dildos that EF has to offer with no problems as of yet. The anus is just like any other muscle in the body, it can do amazing things once properly trained. They key is taking things slow, starting small and working up and listening to your body. If it hurts, stop and try something smaller until your body is ready for the next step.
As long as you make sure you're within your body's limits and playing safe (plenty of lube and warmup, no numbing agents, willing to call it off whenever your body tells you to) you should be fine. When you're at or near your limits it is vital to go slow and be very attuned to pain.
Thinking about this more - yes, there can be objects that are too big for you at a given time. In order to safely play with anal toys you need to be able to recognise this and be comfortable with that, deciding whether you can or should work up to them or just leave them alone. Do note that extreme play should probably be practiced in moderation, because there really aren't any studies on the safety of long-term extreme play.