About every other week, some friend or 1WD reader emails or tweets me a link to Matt Kramer’s “Drinking Out Loud” blog on WineSpectator.com. And about every other week, I have the same reaction after reading it: Kramer writes well, but his conclusions sometimes leave holes large enough that you could drive a steel tank full of Yellowtail through them.
Usually I simply shrug and ignore those conclusions, but Kramer’s piece (titled “Dubious Wine Achievements of Our Time: How smart wine people became boneheads”) published last week struck enough of a nerve that, Shatner style, I just… couldn’t… let… GO!
It wasn’t Kramer’s near-total dismissal of both Bordeaux (maybe he should have called it “Bored-oh?”) as a region (points with which I largely agree, by the way) and Russian River Valley Pinot Noir (points with which I largely disagree) that set me into a fit of head-scratching bewilderment, but his assertion that fine wine recommendations cannot successfully be crowd-sourced.
That latter conclusion flies so forcefully smack-dab into the face of reality that I can only categorize it as head-in-the-sand posturing. Sounds harsh, I suppose, but I see no reason to let something slide when it’s so far off the mark from reality.
Let’s break this one down, shall we?
First, by looking at what Kramer actually wrote, and then at the (potentially invalid) assumptions upon which those statements were based. C’mon, it probably beasts actually working over the next few minutes, doesn’t it?…
Here’s Kramer’s take on the non-wisdom of crowd-sourcing when it comes to wine (emphasis mine):
Not since the old Soviet and Maoist eras have we seen “the people” so lavished with praise and unqualified admiration. Inevitably, the wisdom of the crowd got applied to wine criticism. You see, if enough people—never mind how little they might actually know—post enough of their tasting notes, a crystalline “truth” about a particular wine, or even a whole region or vintage will make itself known. This, in a word, is propaganda. One hundred people who don’t know much about, say, Auxey-Duresses adds up to 100 muddied, baffled and often duplicative conclusions. (You think all those tasting notes were generated in pristine isolation from everyone else’s conclusions?)
And here’s a (classic!) response form the comments of the same post, courtesy of someone who knows something about crowd-sourcing wine reviews:
Matt, thank you for your very perceptive comments on crowd-sourcing. A simple aggregated number is no way to capture such a rich and subjective topic.
-Eric (the CellarTracker guy)
What Kramer missed in his missive is that crowd-sourcing reviews has been already around for years, they already sway individual purchase decisions and are already here to stay (anyone heard of a little company called Amazon.com??). There really is no debate about this – not anymore, and not in any industry that is even remotely keeping up with the times.
There are two big flaws with Kramer’s style of reasoning against crowd-sourcing, because it is based on assumptions and suggestions that very likely aren’t true:
1) Suggesting that wine is somehow so special that it should be immune from the impacts and influence of the trend towards crowd-sourced recommendations is not only a fool’s bet, but it’s a bet that has already been lost. Crowd-sourcing is standard-fare; it’s not tomorrow’s innovation, it’s today’s reality. Just ask the guy who saw the crowd-sourcing phenomenon’s potential for wine almost ten years ago – Eric Levine, author of the above-quoted comment and founder of Cellar Tracker, the home of the largest volume of wine reviews and tasting notes on the planet.
2) Assuming that the make up of the crowds contains no wine knowledge is to ignore the math. If the numbers are high enough, then the reviews most probably contain input from people who have enough relevant wine knowledge to offer a viable, educated opinion; yes, they contain crap as well, but that’s okay because…
3) The crowd itself will police to a large extent which individual voices will carry the most weight. To illustrate this, we need to go back to Kramer’s article for a moment (don’t worry, it will make sense in a minute):
When it comes to fine wine, there is no wisdom of the crowd—that’s not only a delusion, it’s a pernicious delusion. When it comes to fine wine, there is only informed opinion, never mind whose. Such informed opinion can come from anyone.It doesn’t have to be a professional critic. We’ve all met plenty of impassioned wine lovers who have devoted thousands of hours (never mind dollars) to the subject. We should all listen closely to such people. I sure do.
But we must first decide that they are indeed informed. We must first confirm that they have, based on demonstrated experience or exposure, a basis for their opinion. It’s not enough that they are part of the “wisdom of the crowd.”
This is where Kramer actually got it right, though I suspect not in the way that he intended.
Informed opinions are still the lifeblood of quality recommendations. That’s as true for wine as anything else. And you know who determines which opinions are informed these days? The crowd. The crowd is the lifeblood, it is the knowledge base, it is the expertise and the deciding vote; and it is the collective and real wisdom of that crowd that determines which voices are creamy enough to rise to the top.
It’s true for electronics, it’s true for appliances, it’s true for restaurants, and it’s sure as sh*t true for wine now. If you’ve used a vacuum cleaner for the past seven months, and it’s one that I’m considering buying, then for me you’re an expert on the practical use of that vacuum cleaner. If I taste a sh*tload of wine every year then if I do it well and provide valuable output from that, I will be viewed by a lot of people as an expert. Within Cellar Tracker, you have someone like Richard Jennings, who is one of the most prolific contributors there, and with whom I’ve traveled and have seen firsthand that he knows what he’s doing… well, CT in its crowd-sourced wisdom has anointed him an expert, and a lot of people are influenced by what he has to say as a result.
Now, we shouldn’t ignore something else her e- the fact that the crowd’s voice in total has the ability to sway a lot of opinions. It’s the reason why Amazon reviews summaries are so powerful, and why CellarTracker.com aggregate reviews are so influential with individuals at point of sale. In a way, the crowd both makes the expert and is the expert, or at least has an expert’s influence – as evidenced by a recent conversation I had with a group of Millennial wine buyers, who told me that when they’re in a wine shop they often grab their smartphone and Google a wine specifically to see how many stars that wine received from the crowd on Cellar Tracker. If the number of stars are low, they often skip the wine entirely.
Like it or not, that’s (successful) influence.
Or maybe those near-three-million reviews and near-230K users in Cellar Tracker are actually all bunk, and nobody really pays any attention to them.
I know which side I’m betting on, anyway…
42 thoughts on “Fine Wine Reviews Can Never Be Crowd-Sourced! Oh, Wait, They Already Have Been… For Years!”
I completely agree. I have just recently started following experts' blogs to broaden my knowledge base and learn more about something I love. For the most part, however, Cellar tracker is the only thing I check when buying a wine and it has been that way for years.
Dusty – operative phrase there being “for years” :).
I usually enjoy Kramer's take on issues, but this is just over the top elitist bullshit. An opinion formed when 'tasting' 50 some wines 'blind' at 11:00 in the morning has very little to do with the reality of serving a wine to DRINK with dinner. If Kramer thinks that you have to be some sort of 'expert' (how on earth would that be defined?) to taste/drink a bottle of wine and form an opinion on it that is worthy of being shared or considered is, in my opinion, the antithesis of what wine consumption is all about. What a douche.
Masi – ouch! Harsher words than I had, though I respect the position from which you're approaching it. And there is definitely a “hey, you kids, get off my front lawn!” aspect to that article.
Seriously? Only people that “know” should get a vote?! Methinks Kramer is trying to set homself and a small elite set of wine critics above everyone else. Personally, I don’t have the same palate as Kramer and find that his reviews give me little advice on which to choose wine.
Crowd sourcing is here to stay and I welcome the differing opinions, even from the un-wine-tutored masses.
Imkarenp – For me, the thing is that not only has the crowd model been successful already, it has been successful within wine already, and authoritative voices have come out of it. Already. :) It's like he is really, really, *really* late to the party on this one.
Amazon is the only example of this of course — let's just choose movies for example. I think lots of people appreciate the benefits of things like IMDB (consumer crowd sourcing) and Rotten Tomatoes (critic crowd sourcing — a hybrid!), and yet movie critics are still relevant. His comments are just more defensiveness from folks in ivory towers feeling threatened by innovation…
BTW, recall that CellarTracker's original purpose was simply inventory tracking. The tasting note repository sort of came along as an additional feature for personal use, but has emerged to become the most important asset in recent years…
Sid – totally. Assuming you meant Amazon is NOT the only example, of course :).
Indeed — wish I could edit that typo…
Sid – no worries, it will be obvious in the context of the discussion, I'm sure.
What's the evidence for this:
"CellarTracker.com aggregate reviews are so influential with individuals at point of sale."
Tom – anecdotal, and given to small numbers syndrome I'm afraid, based on conversations such as those referenced in the post and the comments already. Doesn't make it wrong, but not substantiated by formal study, either.
Joe – Very glad to see this follow up to our twitter exchange, because you get out in front of enough people to be heard and spread the word. I'm not sure whether Kramer's third point is out of fear that the emporer will be seen to have no clothes or if he will be disappointed when folks arent interested to look. I'm pretty much on the verge of dropping my subscription to Spectator, simply because of loss of relevancy for me, and the information I want to share. Sure, the solo critics and high-end mags can continue to flog the cult wines and stratosphereically priced juices that perennial access gives them, but for most of the folks that I interact with on a daily basis, those wines are unattainable and someone like The Reverse Wine Snob with his under $20 picks, is a much better source of relevant information. Cross that with the crowd sourced backup ratings and reviews, and it's a pretty good formula for delivery to the daily table.
Todd – thank YOU for cluing me onto this one!
The difficulty with crowdsourcing wine (or film) reviews isn't whether those doing the reviewing are expert or all all have stone palates. It's that everyone looks through a personal lens of what constitutes an ideal wine. If I prefer less ripe, less alcoholic, higher acid, more nuanced Pinot Noir (which I do), then I'm going to agree with Matt Kramer about most Russian River goodies, and go over the moon for something from Ken Wright in Oregon. Which, in turn, the Russian River fanboy will find way too lean, tart and significantly underripe.
The ultimate wrongheadedness of crowdsourcing wine reviews is that they merely provide a consensus of the crowd's palate preference — which may or may not be anything like mine. That's why I prefer to follow specific reviewers with known likes and dislikes. If Dan Berger praises a wine, I probably will, too. Robert Parker, maybe not so much.
Joel – I do not see those as exclusionary, but complimentary. Example: I like Critic A and she disses a wine, but the crowd loves it; could be a good wine to take to a party with a lot of people. There are other scenarios, the point is that it's not an either-or situation to me.
I'm not quite sure that I agree that crowd sourced reviews lead to consensus, but rather to a varied scope of response. If in fact folks with different palates still agree on some level of quality, it's not too hard to parse individual reviews from the group, to derive useful information.
Todd – good point. But then, I think in a way you are saying also that there can be some useful consensus (i.e., such as on overall quality)?
There is always potential for agreement on certain points ( the specific like descriptors, or more Meta ones, as in general quality) but I don't know that crowd sourcing automatically leads to any kind of consensus, unless the product in question is so kick-ass that there is no question. If you see any cheap wines that score in that way, definitely let me know!
As for the hard to get, seriously expensive wines, sure, their criticism can't easily be crowd sourced, the crowd does not have access. On this point Kramer is correct.
Which bring up another question if crowd sourcing covers the lower end of wine, and the iconic critics the iconic wines, what's happening to the "middle-class"?
Todd – oh, no, the middle class of wines are getting screwed now just like the middle class earners! ;-)
I guess we should add here that there *are* expensive, high-end, hard-to-get wines that have found reviews on CT. They obviously make up a tiny percentage of the reviews, but some of them *are* there.
I think Kramer (and his fellow employed soldiers) are trying to justify their existing in a shrinking business. New drinking generations are not buying their magazine or subscribing to point systems. They follow blogs. They listen to shop owners. They go out of their way to learn their own palate. His article sounds like he's scared for whats to come and he's lashing out as a result.
Ryan – to some extent, I agree; though my response is here because I'm not a WS member and so couldn't comment on his post directly (which might be telling in its own right).
Now, the other side of the question is when a wine is rare enough or expensive enough that not very many (therefore a LACK of crowd) have reviewed. Case in point: How many reviews are there for a Penfolds Bin 620?
Mark F – in those cases, the scarcity dictates that there will be some gatekeeping. But we're talking about a tiny percentage of even the fine wine world there, aren't we?
True. Good point. Plus… if you could afford or had the chance to taste something that valuable (value = what someone is willing to pay for it), YOU WOULDN'T CARE about anyone else's reviews, RIGHT? ;-)
Mark F – I might, actually; both in terms of sharing/reliving the experience and if it's disappointing commiserating with others! :)
I've always found Cellartracker reviews (especially from people I know, follow and feel align with my tastes) to be more correlative of a wine I might like than a WS or WA score. I've been burned far more by Jay Miller 92 pointers than I have by Cellartracker 92s. It's not a phenomena limited to wine consumption – if I'm looking to buy a new product, Amazon.com reviews are the first place I go. I trust real world user testimonials far more than, say, a Consumer's Report write up.
I'm not a huge "new media" guy who thinks it's the panacea of human evolution, but as pertains to consumer education, I think crowd reviews are not just the norm, they have been for quite some time.
In a similiar vein, Amazon thinks enough of their societal impact to tout online customer ratings as one of their hallmark "inventions" in their new line of tv ads.
John – fantastic summary of the spirit behind this post. And you're totally right, it's not a panacea, but no question it's useful, relevant, and – more importantly – the norm. Cheers!
Awesome, as usual. Thanks for calling out the most dangerous fallacy of all — and the one that informs so much silly writing and posturing about wine — is that wine is somehow different as a category than everything else, and defies the laws of gravity, crowdsourcing, etc etc.
Thanks, Sasha. Gotta admit, that's one of the things that drives me nuts about the wine biz.
I have said it before that CellarTracker! is a great resource for bringing together consumer opinion on wine after they are in the market with each tasting note adding to the tapestry of what people think. There are plenty of astute palates and impressive cellars as part of that community. The argument against the survival of critics misses a key element: The basic contribution of either a critic or even a 'plugged in' wine shop owner/buyer is they will taste before those wines get into the cellars of anyone. With the myriad choices of wines that come out during any year having somebody with a palate you trust or at least understand can offer professional guidance.
thanks, doug – I am certainly not arguing against the survival of critics. I would argue against the survival of ivory-tower-style opinions and pronouncements, though, not just for wine but *any* topic. But I think there are roles for critics to play regardless, for those who can adapt to that changing landscape. I mean, in some sense RJ (who I cited in the post) and I are both critics – we are just not what most would call traditional ones. Cheers!
Richard Jennings is a friend of mine as well and I highly respect his palate (I even include him on the community resource page of my magazine). Adapting to a changing landscape, indeed! The main reason I started publishing is I got tired of reading the major critics write about a wine months after I blogged about it :) My 'tower' is mostly cardboard, and thankfully very little styrofoam!
doug – mine, too! :)
Maybe someone can answer this… when did the wine rating publications start? '78 with the Advocate right?
In any case it was a while ago and they haven't evolved much… at all. With the advent of blogs, CT, smart phones, social networking and so on it has been (for some time) much easier for folks to be in touch and obtain crowd-sourced reviews. This by no means makes publications irrelevant, but they do need to figure out how they are going to adapt in this information age. It appears that they are having a hard time with it – hence pushing back against crowd sourcing and the ivory-tower approach in some instances.
If your palate happens to align with any of these publications then they can still provide a benefit to you. OR you can just read it for the articles ;) I will say, trying to treat wine as if it is different and in a category of its own is silly. At the end of the day wine is a consumer product. The emotion you can attach to wine can make it a special thing, but you can say the same thing about the first car you bought.
Well said, Mike.
To treat all consumer goods as equal is as extreme as saying wine is "sooo" special? Wine is different, not special(!), but definitely different. As are restaurants. And music and books! With its big range of styles and insane amount of choice, wine is more like music and books than vacuum cleaners.
As an example – if we source music purely based on views on Youtube (and do NOT segment) – you will not be listening to too much beyond Justin Bieber and PSY. Both of which I adore btw! But I also like AC/DC "Shoot to Thrill" live in Argentina, Janele Monae's performance on Lopez tonight , and Elina Garanca singing Mozart. All of which, if I apply a simple linear scale based on Youtube views with Bieber at 100pts, will score less than 1pts out of 100pts!
In fact, by crowd-sourcing logic, Bieber's "Baby" is probably the best song that has ever been produced? 700 million views can't be wrong? Some of the most enjoyable books ever written are probably not the ones that have sold the most. Some of the most beautiful movies ever made probably did not have the highest box-office receipts (Star Wars Trilogy is an exception of course …).
When it comes to matters of taste, it is extremely personal. Which by definition, means that whether what you are buying is "popular" is not relevant? Crowd-sourcing is simply a measure of popularity. Buying a bottle of wine based on a CellarTracker recommendation to take to a party, is equivalent to buying a music CD to take to a party based on Youtube views? It might work, it might not …
The fact that CellarTracker exists and is influential does not make it "right". Neither am I suggesting that the tasting panel at WS is "right"! Segmentation is what is missing.
Tai-Ran – I agree with you, actually. Segmentation in complex markets like wine is key. But taken overall, wine isn't special in that it's not immune to the trends impacting very other consumer goods area.
I'd just like to echo both Todd and Tai-Ran and say that there is room for both but that we need to be conscious of that squeezed "middle class". It seems to me that wine is undergoing a similar trajectory of *certain* other cultural-consumer products. I sometimes think of wine as being akin to Shakespeare. When Shakespeare was writing and performing, the goal was to be able to entertain all segments of society in different ways, from the nobility to the Groundlings. It was popular entertainment. Likewise, wine has always served multiple tables, from the highest nobility to the lowest peasant farmer. Like Shakespeare, who has been deified (with the result that most productions are stale and boring), wine has been elevated to a level of cultural cache that marks people as sophisticated members of society. Unfortunately, like with Shakespeare, a lot of "sophisticated" viewers/drinkers don't really know how to *contextualize* what they are seeing and/or tasting outside of their own sensory experience. Thus, there is strong value to critics, who have the job of doing that. Additionally, many wine critics have access to wines that we mere mortals do not have access to, unless we mortgage our houses to buy those bottles!
There is also strong value to understanding how to use crowd-sourcing when making a purchasing decision. And this is, I think, the point of your post: that the world needs both aggregate numbers to help guide us, *and* informed critics to help contextualize things beyond our personal experiences. Thank you for a very stimulating post!
Thanks, William. Really enjoyed and appreciated your comment. Yeah, part of the point is definitely that we need both, with the other part being that Kramer is way off-base in saying that the crowd portion doesn't matter. Cheers!
Felt compelled to add this – even important astronomical discoveries are crowd-sourced!
Comments are closed.