I believe you can have too much of anything. Life needs balance. If you're neglecting your health or responsibilities, it may be too much, and you should cut back. But at the same time, you should have enough sex to feel fulfilled -- you can have too much work or whatever in you life that cuts into your sex time, too. It's like those mice who were given a button to push that gave them reward pleasure -- they stopped eating and just pushed the button. (Reference.
That said, I think most cases of being accused of "too much sex" are really a moral imposition, not an actual case of too much. I also think the idea of cheating gets conflated with sex (i.e. all those case of "sexual addiction" where it's really just "I got caught cheating", which has nothing to do with sex in and of itself, completely separate issue), as well as other non-vanilla aspects of sex (porn, BDSM, etc.).