I think it all does factor into my rating of a product, but not by much.
For instance, I just wrote a review on the Kissa
and it still got 1-star, even though it is made of glass and plastic.
I think it really depends on how the material affected my use of the product. Was it glass, but it was really hard to clean and painful and dulled in vibration? Well, sounds like despite the glass, the product was a dud for me.
Was it a lower-safety material, but it performed wonderfully and has become my go-to toy? Well, 5-stars it is!
I always mention the material in the review, because I do think it is important for people to make their purchasing decision with all the info. And rather than just a material, I get to comment on what I thought about how it performed with this material... if that makes sense.