We do sometimes, just depends on how I'm feeling. Usually I don't feel up to it until close to the end anyway, when my flow is really light. My partner has absolutely NO problem with it, he's even said he'd happily give me oral on my period.
I had never even remotely considered having sex on my period until my fiance suggested we try it. We talked about it and set some ground rules. Sex off my period is definitely better but it's mostly the fact that my brain can't get over the fact that I am on my period and having sex. It does not bother my fiance at at all.