Glass and plastic are stiff, so they can poke and be uncomfortable (perhaps even dangerous). Rubber, TPR and jelly can't be fully cleaned. Silicone can be sanitized and is flexible, so silicone all the way..
Silicone indeed (in case you needed another endorsement for silicone). Not only for the reasons that ScottA mentioned, but also because silicone absorbs your body temperature after a short while, so it feels comfortable.