The day I received the results from the 2009 Decanter World Wine Awards, I also received an e-mail from Karl Storchmann from the American Association of Wine Economists announcing the newest publication of the Journal of Wine Economics (Vol 4, No 1).
The lead article in this issue of the journal is about the wine judges and their reliability in judging wines. This was the perfect contrast reading to the long list of award winners from Decanter. Again Robert T. Hodgson has summarized some more research about the topic in question.
The title of the article is “An Analysis of the Concordance among 13 U.S. Wine Competitions”. Hodgson followed over 4,000 wines entered in these competitions of which 2,440 wines were entered in more than three competitions. 47% of these wines received gold medals in one show but 84% of these same wines received no medals at all in the subsequent shows. These results indicate that winning a Gold medal is greatly influenced by chance alone. Or to put it another way: Wines which were deemed of an extraordinary quality by some wine judges were assessed as very ordinary by others.
In come the Decanter World Wine Awards 2009 results. What does the study say about these awards. Nothing, of course. My skepticism about wine shows, however, is rather stronger then before.
One may ask the question are the Hodgson findings about wine shows in the U.S. applicable to shows such as Decanter? Can we safely conclude that some of the Decanter award winners would be seen as ordinary by other wine judges at other shows, and that some of non-winners could easily win a Gold somewhere else. Or could the Decanter wine judges replicate the tasting with the exact identical findings? Or what would their “margin of error” or inconsistency be?
What Hodgson’s study also tells us is, that wine judges are just human beings who are inconsistent and unpredictable. But we might rightly assume that they are trying to do the right thing but fail in this from time to time. The judges at the wine competitions surveyed, though, seem to have a shared idea about the wines they do not like and therefore win no awards at all. That’s at least something.
A few wine blogs have taken up the issue as well. Alder Yarrow for instance writes on Vinography about the AAWE paper and calls for a stop of the proliferating state wine shows. He admits, however, that the shows surveyed where, and I quote, “essentially the largest and most prestigious wine competitions in America”.
What does this tell us wine consumers? Well, wine quality and medals won at wine shows are not “orthogonal”, the presence of one doesn’t imply the presence of the other. Therefore, trust your own taste and buy what you like, possibly after tasting and let the wine shows be wine shows.
How about the vintner? Well, it is very nice for any wine producer to be recognized for outstanding quality and performance. So it is nice to win a medal at a wine show or getting a five star winery rating from James Halliday for instance. But it is not the end if this is not coming along. Most important is that ordinary people like to drink and buy your wine; the rest takes care of itself.
Cheers folks, trust your own judgment and taste as many wines as you can.
The Man from Mosel River 
Dear Rainer,
well writen; you “speek me out of my soul”. “In vino veritas” offenbart sich immer und ausschließlich für den, der wein trinkt, in dem Augenblick, wenn er ihn persönlich trinkt. Egal, was andere vor ihm über “diesen” Wein gesagt oder geschrieben haben. Aber es ist ja nicht derselbe Wein, denn nicht nur jede Flasche, sogar jeder Tropfen ist individuell. Allerdings mag “dieser” Wein besser schmecken, wenn ihm jemand vorher eine “Goldmedaille”, 100 Punkte, 5 Sterne oder 3 Becher verliehen hat und man das weiß. Die Erwartung bzw. Vorfreude schwingt dann mit.
Freuen wir uns also auf möglichst viele und hoffentlich gute Weine. Ich beherzige deinen Rat gerne und natürlich immer wieder!
Cheers
Thomas