I think everybody needs Downtime.
I was raised an Only Child, so I was used to spending a lot of time alone. My husband had a brother (passed away), but they were almost 8 years apart, so he spent a lot of time alone as a kid, too.
I need my alone time. Although I loved raising my babies, it was difficult at first, because they need you all the time! Add that to our first baby being very colicky and needing to be held all the time, I was getting very depressed. (Add a difficult labor and an unwanted C Section and you have a recipe for PPD, which I had.) But, I kept at it, because I knew she needed me more than I even needed my alone time. We got through it, and it got easier with each kid.
But, I still need my alone time. Sometimes I LOVE company, or to spend time with my kids and my man. But, sometimes not. I'm OTR right now, and my middle daughter came into my room all sunny and talkative this morning (both our dogs were howling at the Tuesday morning sirens and she wanted to talk about it) and after a few minutes, I was getting fidgety and really wanted to be alone. But, I stuck with her for about a half and hour and then let her know my head was hurting and I had cramps, so she moved on to other things.
Our youngest has Aspergers and usually wants to be alone. We often have to prod her to socialize. Of course when she wants company, it's a different story.