Don't let your ex dictate your sex life. Your sex life is your business and is no longer any of his business. If he can't handle that, then perhaps keeping him as a friend just isn't going to happen, and that's okay. In my experience, exes don't make great friends; there's too much of a history between you guys, and chances are, if he cares about who you're having sex with still, then he's not over you, and there will be hard feelings as a result. Don't not do what YOU want because of what he wants. How long will a friendship like that sustain itself, anyway? Eventually, you're going to get fed up with making compromises on his behalf, because after all, he's not your boyfriend anymore. So as the resentment towards him grows, you'll probably eventually do it anyway, and if he no longer has any control over that aspect of your life, then he's going to get upset. Either way, it doesn't look good as far as your "friendship" goes. Why not just skip over all that drama, and do what feels right to you?

The best thing for people to do when they break up with someone, is get some space. You are not likely to be friends until you've both had the chance to get over each other and stop worrying about who the other one is fucking.

Now, if the issue is the mutual friend, then that's going to be tricky any way you cut it. I don't entirely understand why your ex even knows about your desire to get into some kinky experimentation with this other person, because like I said, your sex life is no longer his business. But you're all adults (I hope), so make the choice for yourself. But to be honest, I'd take a step back from your ex and enjoy the space and the breathing room. It will be better for you both, and is the best chance of having a friendship in the future, once emotions have cooled down, and everyone has moved on. You guys weren't sexually compatible, and now you have the freedom to explore, so take it. Otherwise, you might as well be back in that unsatisfying vanilla relationship with your ex, because you're still not exploring your sexuality and desires with him hovering around.