We all read that Barry Estabrook article in The Atlantic a few months ago, right? The one that made a solid case that organic food production can feed the world. I’m sold on that point but in my mind the question remains: would an organic diet be delicious?
Food is about choices that sacrifice one variable for another. When producers choose to limit chemical inputs (inorganic fertilisers and pesticides) my back-of-the-envelope guess was that taste often suffers. I know from gardening at home that when I select a certain seed because I’ve read it produces uncommonly delicious tomatoes I have to accept that they won’t be “heavy croppers” or resistent to four types of wilt.
I’m just as unsatisfied with my random guesses as you probably are so moved on to a more objective test.
I’ve noticed over the five years or more that I’ve been a Cook’s Illustrated subscriber that their monthly taste tests of ingredients or prepared foods often feature a few organic options. I wanted to get a sense of the overall record between the organic and conventional options.
Jack Bishop, Editorial Director of the PBS show and Executive Editor of the magazine runs the tasting lab and presents the results on America’s Test Kitchen with Chris Kimball as a sort of taste-test straight man. You can get a sense of the methodology from this clip of a parmesan taste test.
Of the nearly 400 taste test results posted on their website (a subscription to cooksillustrated.com grants access to their top-notch recipes, equipment reviews, and taste tests at an affordable price) in late February, 2012 seventy-five of them returned results when I searched for “organic”.
The taste test archives stretch back to January 2002 and span categories from pantry items (a lot of these) and prepared food, right to dairy products and meat.
When the taste test results are printed they’re listed in order of preference and given a recommendation level that is either “highly recommended”, “recommended”, “recommended with reservations”, or “not recommended”.
I put the results from the 75 tests into a spreadsheet and to make comparisons using these two variables (rank and recommendation) at the same time easier I converted each result to a score out of 100. The first–easy part–is twenty points for each rank (so 80 for highly recommended, 60 for recommended, 40 for recommended with reservations, and 20 for not recommended). The other twenty are assigned with the formula 20 – ((x/y) * 20) where x is the rank and y is the number of entries in the test.
As an example let’s say that a particular brand of pinto beans was ranked second in a field of four and fell in the recommended category. That’s 60 points for recommended and 10 for second place for a total of 70.
Or pretend that a certain cottage cheese was not recommended and finished second-last in a field of seven. That would be 20 for the bottom category and 2.9 points for sixth place for a total of 22.7.
Obviously, in a particular taste test the scores drop off quickly and the recommendation category is much more important compared to the rank, which are basically just tie-breakers.
|Total Items (75 categories)||431||126|
|Categories won by Average||39||36|
|Categories Won Outright||60||15|
|Best Option Above Avg||73||43|
|Best Option Not Recommended||2||14|
|Best Option Last in Category||1||5|
On the top line the organic food options (126 of them) scored an average of 49.23 points versus the conventional food average (for 431 items) of 54.62. The scores remain fairly close by category where the organic average was higher in 36 of the categories versus 39 where the non-organic had a better average.
Only looking at averages gives us a good picture of what all of us would get, given lots of options and little information or care about our shopping choices. But there is a limit to how realistically that represents outcomes for individuals. After all, if we’re making chili we don’t use parts of four different cans of kidney beans. We pick one brand–sometimes with information like the recommendation from an authority like CI–and either enjoy or suffer the results.
So, for the rest of the comparisons it will help to picture an olde tyme store where Jack Bishop is the grocer. You ask him for what you want (filet mignon, whole wheat lasagna, olive oil, whatever), tell him whether it should be organic or not, and he’ll give you what he thinks will taste best. (We don’t have these stores but we do have smartphone apps that can lend the same expertise to grocery shopping.)
Pretend you’ve moved, are restocking your kitchen, and ask for one of the best from each of the 75 food categories; 60 will be conventional and only 15 organic. If you say to Jack, “give me your finest organic option” he’ll be able to beat the average score about 57% of the time–naturally the flip side of that is that 43% of the time the best organic option is below the average for the category. Unfortunately, 14 times he’ll probably hesitate before sliding something across the counter that ranked Not Recommended and finished in the bottom third of its category. Five times the best he’ll be able to do, if you want organic, will be the option that placed dead last in its category. The non-organic side only had these last two distinctly negative showings twice–both in frozen fruit categories.
I’d worry just as much about the number of times that choosing organic precludes a great option. Twenty times non-organic products received the rare “highly recommended” designation but that only happened once on the organic side. From cheese to chickpeas; yogurt to butter that’s a lot of instances where you’d be excluding the remarkably delicious ingredients by eating strictly organic food.
We haven’t even considered the more than 300 taste tests where there wasn’t any organic option good enough, or available widely enough to make it into the tested field.
I suspect for some, these results will come as no big surprise: They assume organic food is for hippies and rabbits, not serious eaters. Others who lead an organic lifestyle and are evangelical about that choice will be up in arms–and please throw out your objections in the comment section and I’ll respond in a future post. I was (just slightly) surprised both how poorly organic did in the aggregate and conversely how many times an organic option floated to the top of a large field. Solid confirmation, I suppose, that when you put taste and flavour first, other characteristics suffer and that choosing delicious food requires a good amount of care.