I voted yes, but I suppose there are times when it might not be ok. But in general, I don't see a problem with being friends with an ex. In fact, I'm still quite close with an ex. We chat often.
If you think it's causing trouble in your new relationship, then it could be that you're still hung up on the ex, or it could be an issue of lack of trust or other issues with your new relationship. Very rarely is being friends with an ex the reason that a new relationship breaks up, in my opinion. If someone else can get between you two, then there were mostly likely issues to begin with.
Now, if the ex is bugging you all the time, or messaging inappropriate things or doing things you don't like, then un-friend them, obviously. But, a healthy friendship with good boundaries and a mutual respect, I don't see a problem with that. When you aren't with someone anymore, I don't see the need for them to vanish. So, it didn't work out. There could still be friendship potential there.