Do you think that it takes away from a toy if it's light weight?

Depends on the toy. Vibrators have mechanical parts, so once it gets lighter than a certain amount you know they're scrimping on the mechanism or it uses small batteries, but in most cases it's a matter of design rather than weight.

Note that hollow (air-filled rather than toys with a structure inside) toys usually don't work well.