Food With Legs Rotating Header Image

Organic Food Taste Test

Cook's Illustrated covers and the taste test of canned whole tomatoes.

We all read that Barry Estabrook article in The Atlantic a few months ago, right? The one that made a solid case that organic food production can feed the world. I’m sold on that point but in my mind the question remains: would an organic diet be delicious?

Food is about choices that sacrifice one variable for another. When producers choose to limit chemical inputs (inorganic fertilisers and pesticides) my back-of-the-envelope guess was that taste often suffers. I know from gardening at home that when I select a certain seed because I’ve read it produces uncommonly delicious tomatoes I have to accept that they won’t be “heavy croppers” or resistent to four types of wilt.

I’m just as unsatisfied with my random guesses as you probably are so moved on to a more objective test.

I’ve noticed over the five years or more that I’ve been a Cook’s Illustrated subscriber that their monthly taste tests of ingredients or prepared foods often feature a few organic options. I wanted to get a sense of the overall record between the organic and conventional options.

Jack Bishop, Editorial Director of the PBS show and Executive Editor of the magazine runs the tasting lab and presents the results on America’s Test Kitchen with Chris Kimball as a sort of taste-test straight man. You can get a sense of the methodology from this clip of a parmesan taste test.

Of the nearly 400 taste test results posted on their website (a subscription to cooksillustrated.com grants access to their top-notch recipes, equipment reviews, and taste tests at an affordable price) in late February, 2012 seventy-five of them returned results when I searched for “organic”.

The taste test archives stretch back to January 2002 and span categories from pantry items (a lot of these) and prepared food, right to dairy products and meat.

When the taste test results are printed they’re listed in order of preference and given a recommendation level that is either “highly recommended”, “recommended”, “recommended with reservations”, or “not recommended”.

I put the results from the 75 tests into a spreadsheet and to make comparisons using these two variables (rank and recommendation) at the same time easier I converted each result to a score out of 100. The first–easy part–is twenty points for each rank (so 80 for highly recommended, 60 for recommended, 40 for recommended with reservations, and 20 for not recommended). The other twenty are assigned with the formula 20 – ((x/y) * 20) where x is the rank and y is the number of entries in the test.

As an example let’s say that a particular brand of pinto beans was ranked second in a field of four and fell in the recommended category. That’s 60 points for recommended and 10 for second place for a total of 70.

Or pretend that a certain cottage cheese was not recommended and finished second-last in a field of seven. That would be 20 for the bottom category and 2.9 points for sixth place for a total of 22.7.

Obviously, in a particular taste test the scores drop off quickly and the recommendation category is much more important compared to the rank, which are basically just tie-breakers.

Conventional Organic
Total Items (75 categories) 431 126
Average Score 54.62 49.23
Categories won by Average 39 36
Categories Won Outright 60 15
Best Option Above Avg 73 43
Best Option Not Recommended 2 14
Best Option Last in Category 1 5
Highly Recommended 20 1

On the top line the organic food options (126 of them) scored an average of 49.23 points versus the conventional food average (for 431 items) of 54.62. The scores remain fairly close by category where the organic average was higher in 36 of the categories versus 39 where the non-organic had a better average.

Only looking at averages gives us a good picture of what all of us would get, given lots of options and little information or care about our shopping choices. But there is a limit to how realistically that represents outcomes for individuals. After all, if we’re making chili we don’t use parts of four different cans of kidney beans. We pick one brand–sometimes with information like the recommendation from an authority like CI–and either enjoy or suffer the results.

So, for the rest of the comparisons it will help to picture an olde tyme store where Jack Bishop is the grocer. You ask him for what you want (filet mignon, whole wheat lasagna, olive oil, whatever), tell him whether it should be organic or not, and he’ll give you what he thinks will taste best. (We don’t have these stores but we do have smartphone apps that can lend the same expertise to grocery shopping.)

Pretend you’ve moved, are restocking your kitchen, and ask for one of the best from each of the 75 food categories; 60 will be conventional and only 15 organic. If you say to Jack, “give me your finest organic option” he’ll be able to beat the average score about 57% of the time–naturally the flip side of that is that 43% of the time the best organic option is below the average for the category. Unfortunately, 14 times he’ll probably hesitate before sliding something across the counter that ranked Not Recommended and finished in the bottom third of its category. Five times the best he’ll be able to do, if you want organic, will be the option that placed dead last in its category. The non-organic side only had these last two distinctly negative showings twice–both in frozen fruit categories.

I’d worry just as much about the number of times that choosing organic precludes a great option. Twenty times non-organic products received the rare “highly recommended” designation but that only happened once on the organic side. From cheese to chickpeas; yogurt to butter that’s a lot of instances where you’d be excluding the remarkably delicious ingredients by eating strictly organic food.

We haven’t even considered the more than 300 taste tests where there wasn’t any organic option good enough, or available widely enough to make it into the tested field.

I suspect for some, these results will come as no big surprise: They assume organic food is for hippies and rabbits, not serious eaters. Others who lead an organic lifestyle and are evangelical about that choice will be up in arms–and please throw out your objections in the comment section and I’ll respond in a future post. I was (just slightly) surprised both how poorly organic did in the aggregate and conversely how many times an organic option floated to the top of a large field. Solid confirmation, I suppose, that when you put taste and flavour first, other characteristics suffer and that choosing delicious food requires a good amount of care.

Similar Posts:

Share this post: More Food With Legs:
Posted in: Culture of Food, Food Shopping.

6 Comments

  1. I am an organic farmer turned foodtrucker turned diner owner and I have to say that no matter what your numbers and charts say, there is absolutely no way a chicken kept in a cage and fed medicated feed will ever taste better or produce tastier eggs than a chicken allowed to run free, eat bugs, scratch the earth and eat natural food, same goes for goats, cattle, sheep and turkeys. There is no way a piece of fruit sprayed with guthion will ever taste better than one that’s not, there’s no way a tomato fertilized with chemicals will taste better than one fertilized with compost tea, eggshells and manure….then there’s, health, environmental impact and sustainability to consider….organic is a better choice all around. :)

  2. pat anderson says:

    Really interesting, David! I watched the video as well. In the video, they make a point of calling out the price difference — I don’t know if price is one of the qualities they examine when rating food? If it is, given that organic foods are frequently priced higher than conventionally farmed, could that be one of the differentiators that drove it down in terms of recommendation? Or is taste the sole arbitor?

  3. foodwithlegs says:

    Thanks for commenting, Regal. I really appreciate your perspective and passion.

    So, I wanted to be careful to respect the copyright of Cook’s Illustrated and not print the list of food tested and the results. But, no fresh produce is involved here and I did say that the two categories where conventional was blown away were two of three frozen fruit categories. If you want to picture the food involved think pantry items (including quite a bit of canned vegetables), pasta, some dairy, and a bit of meat.

    On the meat front, and actually in general, remember that non-organic does not necessarily mean CAFO, giant feedlots, corn-fed, and no attention to selecting heritage stock. There are some near-organic, small producers, that don’t get certified for a host of reasons. I don’t know for sure, because they tasted American brands, but it looks from the prices like the massive, value operations lost the two chicken taste tests.

    You’re right that there are more factors in the decision but to keep things clear I wanted to focus on taste–the others have been better covered by others. “Numbers and charts” is a fair characterisation of the medium I chose but please remember that the message is about real people (who are experts tasters) judging what they think is most delicious.

  4. foodwithlegs says:

    Interesting point, Pat, thanks for raising it. I have to admit that on one hand CI never explicitly says they don’t take price into account when doing the ratings I’m 98% certain they don’t. These are the reasons why:
    1. The results include a comment for each item that explains where they placed and why. I have never seen one refer to price (and after researching this post I’ve read a lot);
    2. They stress that the tests are blind and to me that means isolating taste as the only variable; and
    3. Prices are printed and from what I can tell they aren’t a drag on the ratings.

    I’m working on a second post that looks at how much organic food costs (based on those prices) so look for more in that post.

  5. sockpuppet says:

    I would argue that are tastebuds have been altered by years of excessive sodium, preservative, artificial coloring and flavoring –laden processed products and that we no longer can taste “real” food without thinking it tastes bland. In actually, it takes a bit of time to adjust. Once you do, you can’t go back to the artificial products. This may be a bad example, but I used to love twinkies. I stopped eating them for healthier choices, and now when I buy one about once a year or so, it just tastes fake and plastic and nothing like real food.
    On another point though, fresh organic products like produce and eggs are hands down substantially tastier than over-produced, modified produce. There isn’t a person I know that hasn’t tried it and noticed the huge difference. So if they are rating mostly processed food, it’s not a fair comparison.

  6. foodwithlegs says:

    sockpuppet: You make an interesting point about how our palates become acclimatised to certain flavours and desensitised to certain additives but you lost me after that.

    These were all apples-to-apples comparisons–except not really because none of them are for fresh produce. A can of beans is equally processed whether the beans are grown organically or not. My idea here is to answer a question by combining information from the answers to many other questions. So Cook’s Illustrated asked “Which whole wheat lasagna tastes best”, “Which brand of mirin tastes best”, “Which ketchup tastes best” etc. and by noting how the organic options did I was able to start answering the question “Does a particular organic food taste as good as its non-organic counterpart”.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>