If you divorce someone for being infertile then, in my opinion, you didn't love them as much as you should've and should never have married them in the first place! If the only problem is your partners infertility then you have choices; adoption, sperm/egg donor, surrogate, etc.
I choose ALL would be grounds for divorce EXCEPT extreme weight gain, porn addiction, committed serious crime and infertility. The serious crime thing is a little iffy. Example, say if my man killed/harmed someone protecting me or family, I would stick my his side 25 to life