Don't worry about how long others took. You need to take the amount of time you need. If you're not ready, comfortable and confident it won't be fun.
Remind him that it's in his interest to make sure you're ready, because if you love anal he'll get it more often. Practice playing around back there with fingers and toys, learn how to relax and enjoy it, then bring him in with fingers and toys. Only when you're comfortable with anal play and him giving you anal stimulation should you try penis-anal sex. Have a backup plan to so he knows he'll still have some fun if you call it off partway, then don't be afraid to call it off partway if you're not ready.