I trim downstairs fairly regularly (not with clippers, I just shave around the edges to sort of keep some order....)
Otherwise I don't, unless I fancy it. It's really up to you. I'll have days when I drag up and shave everything (except my head), but most of the time I'm fairly hairy.
Top tip: Shave off your side-burns as high up as you can get away with; when they start to grow back it'll look like stubble