Silicone is easy to acquire, safe, and simple to mold with the right materials plus it's affordable. I'm actually super interested in making my own toys!
I'd love to make a wood toy too, just gotta find a body safe sealant, which obviously exists since there's plenty of wooden toys out there.
I really don't see any danger in making your own toys as long as you aren't messing with unsafe chemicals and plastics that you shouldn't be putting in your body anyways!