I buy locally and organically. I am lucky enough that I live in city where everyday of the week there is a farmers market of local/organic farmers, butchers, cheesemakers and such in different parts of the city so I can stop by any one of them anytime I need something. It is also a small city so nothing is more than 10 or 15 minutes away from where I live.
It is a bit more expensive but I find that it is something that is important to me so I made changes in my life to make sure I can budget it in. As for herbs, lettuce, green onions and things like tomatoes, strawberries, and summer squash I grown on my own.