You're already late, in my book. The right time is from the get-go, for me, especially if your other partner is definitely the primary one, and you don't have any intention of letting this new person interfere with the primaryness of your other partner.
Being the third is something that I feel like you need to go into knowingly...
Pretty much all of my experiences in non-monogamy, everyone knew about each other from the beginning.
I think secrets are the enemy of functional poly.