If the voltage, polarity, and plug size all match, it will work fine, but it's not a good practice to swap chargers without carefully reading all the specs in question. If the voltage is right, but you forget about polarity, it'll fry the toy in seconds. I've had it happen with other electronics.
Also, Lelo warns specifically against mixing the charger for the Smart Wand with its other products. Seeing as they ship the Gigi with a charger that doesn't even match the specs in the user manual, I'm going to take anything they bother to warn against very very seriously.