I wear matching sets when I'm seeing my partner but if I lived with someone or even even saw them on a more regular basis I wouldn't match every time. I own a lot more panties than bras so if I wanted to match every day I'd be doing a lot of laundry.
I don't buy sets, but I do like matching complimentary patterns, fabrics, colors, etc.,. Separates are usually more exciting than sets, anyways. I've found that most of the time, either the top or bottom is not as cool.