Aside from toys made of hard plastic (which is usually not porous), I generally try to stick with silicone, or possibly glass and metal (though I've yet to actually try the latter). It's mainly out of convenience - they're just easier to care for, and can last forever. Though I do have a couple toys made of TPR and TPR silicone.

I guess I just like knowing that when I've cleaned my toy, I've pretty much killed any bacteria that might have hopped onto it, and that none have crept inside. That and more porous materials seem easier to damage. I wouldn't completely rule out a toy based on material, though, if it truly looked like it might be fun - so long as it's pthalate-free, of course. less