Mkay, so I have tried this and I have liked it. It does, however, depend on the pleasure balls/vaginal balls we're talking about.
I have the Couture Collection Eclipse (review is up on my page if you want to know more. I just recently reviewed it) and that was the most fun for me so for. Anything that is weighted is going to give you a really nice feel. Almost like a vibration, but not constant. To me, this is the best inner vaginal pleasure you will get while riding a bicycle!
Another great pair to use is the Luna Pleasure Bead System. They can help you switch it up and see what you like. I would just be weary of using any small balls that can possibly slip out. It did happen once and was an uncomfortable ride home hoping that a ball wouldn't slip out of my pants!