WMC: "Millennials Consume 42% of all wine in America"
The Wine Market Council presented their annual 2016 roadshow in New York in January, and using the above slide announced with fanfare that Millennials are now the largest wine drinking population in America, consuming 42% of all wine and surpassing the boomers with 30% of total consumption. They also said Millennials were consuming 160 million cases compared to 114 million cases consumed for boomers (below right chart).
To many of us in the business the facts appeared grossly exaggerated, but the media ran with the story because it was such a senstational headline. The long-awaited ascendance of the millennial had finally come we were told, and the articles proclaiming the fact hit the wires in waves:
- Millennials Top U.S. in Wine Consumption (CBS News)
- Millennials Come out on Top in U.S Wine Consumption (S.F. Chronicle)
WMC: "Millennials and Boomers Consume the Same Amount of Wine."
"Boomers and Millennials today account for nearly the same amount of wine consumption and Millennials will soon account for decidedly more consumption."
So which statement should we believe? Are millennials the largest consuming cohort, or are they equal to boomers in consumption? The answer is: Nobody knows.
Rather than retracting the earlier results released in NY in January and providing some clarity, the WMC unbelievably just published different findings and removed the other slides in question. I don't know why they have taken this path. Maybe they were hoping nobody would notice their gaffe and they could just continue on?
Sadly that is the case as of this moment. Nobody noticed the mistake, but the consequences will be felt by the industry the WMC serves because to this day we are still seeing articles published referencing the earlier incorrect findings:
- Low-Lux Wines Draw in Millennial Consumers (CNBC - The Street.com)
Why Does it Matter if the Data are a Little Off?
It matters because you can hear and see Tom Steffanci, president of Deutch Family Wine & Spirits reciting the bogus WMC consumption facts as support for his current strategy in this video (link). I don't know him, but I think Tom would care deeply about being put in the position of reciting bad information to the press.
It matters because two weeks ago a prospect presented a deck to me and wanted me to opine on his strategy for a product. He was targeting millennials and had raised millions of dollars from others leaning on the fact from the WMC that 42% of wine consumption is now from millennials. I doubt the investors in this case would be too happy reading this post.
Reality: Millennials Average About 15% of Consumption, Not 42%
What about the re-engineered consumption statement made in Yountville that positioned boomers and millennials with "nearly the same consumption?" Where did that revised fact come from? I can't find any other research that places millennials and boomers with equal consumption. And in this case, incredibly, the WMC itself hasn't issued any numerical facts supporting their own follow-up conclusion.
Questions Begging an Answer
 |
It was noted during the presentation that there had been a "healthy internal debate" within the WMC Board of Directors regarding the New York findings. But specifically which findings were being debated wasn't disclosed to those attending. The only slide corrected was the one to the left.
This slide restated millennial glasses consumed per drinking occasion from 3.1 glasses per occasion, to 2.4 glasses per occasion. But that didn't resolve the obvious mistake made in their total consumption findings (top chart in this blog).
The information presented in New York was wrong. The WMC changed their conclusion so that fact isn't debatable, but now we are left guessing which findings are right and which are incorrect.
Instead of being transparent about the problems, the WMC elected to simply remove the first slide at the top of the page from the deck. No mention was made of that. That slide and the Total Case Consumed slides just disappeared from the Yountville presentation without comment.
More Questions Now than Before
The inability to support the new total consumption claims with any follow-on hard facts, and the lack of transparency into research practices leaves more questions unanswered now than before:
- Was this a math mistake, a process mistake, or is there a systemic bias in the research techniques?
- If there is systemic bias in the sample, how many years back does it leak into prior conclusions?
- The data set for the cohort consumption finding was from the High Frequency Tracking Study. How many other findings from that data set should be reexamined in this report?
- The slide corrected in Yountville was from the WMC-ORC Segmentation Survey. How many other findings from that data set should be reexamined?
- How did they come to the changed conclusions on total consumption without numeric support? If they had support, why wasn't that presented?
- How can the WMC conclude Millennials will "soon account for decidedly more" without presenting any facts in support?
-----------------------------------------
What do you think?
- What would you do if you were in the shoes of the WMC? How would you handle this?
- Do you believe the old findings, or the new WMC statement on millennial consumption?
- Have you seen any research I've missed that supports the second claim, that Boomers and Millennials are "about equal in consumption?"
- Any other thoughts you have, pro or con are welcome.
- Please join the site in the upper right side of the page.
- Log in and offer your comments for the benefit of the community.
- Please publicize this post on your favorite social media platform
There used to be an oft quoted observation: "Americans would make a fortune about what they don't know about economics." The WMC reports over the years have always been "interpretive" in the broadest sense of the word, but this was clearly a reach and a bit of math that would do me proud (me being the one who scored in the SATs in the 99th percentile for writing and such, and 15th percentile for all things math related. There is so much energy being wasted on worrying about what the millenials are doing, and making further leaps that Boomers will suddenly stop and change their consumption habits as they get older, it makes it difficult to even have a conversation with some marketers. I hope John and company respond to your post, or even better, sit down with you and go through their conclusions, and allow some cogent conversation about them...and then announce a new wave that more accurately reflects reality.
ReplyDeleteThanks for the comment Jim and for logging in. Much appreciated.
DeleteI've believed the "millennial myths" in the wine business as I've called them, have been overstated by prior WMC findings and I've had direct conversations about those concerns, but without any resolution.
That's ok. Research can come to different conclusions on findings within some range, as long as the math is good and any bias can be explained. The findings presented this year in NY were so far out of all other research I can find however, I felt I needed to reengage with the organization to see if something could be done to improve the narrative in the market. It really has been a waste of energy as you note.
In this case, I've communicated directly with people within the Wine Market Council over the past two months regarding this situation, hoping to prod them into clarifying their statements. I believe the difficult discussions were taking place. John Gillespie noted that in the Yountville presentation, but so far outside of the minor correction noted in the blog, there has been no retraction of the prior findings.
In the end the messaging and the facts they want to present are theirs so I certainly don't want to be the arbiter or interpreter of their set of data. They can and I hope soon will do the right thing and help us all understand how they came to their conclusions.
I have a lot of hope in that direction because there are great people in the Board and their Research Council who are quite skilled in understanding and conducting research. I don't think any of them are paid however so I suspect their contributions and oversight in the conclusions have been more limited in the past. They all have full time jobs.
Hopefully the organization will take this as an opportunity to get the research and messaging right and add a component of transparency to help us all better employ their research and findings in decision-making.
The age cohort they use to define Millenials is not in agreement with most research in other fields. Skewing the ages alone skews the data. Hopefully they will right the ship.
ReplyDeleteLiz - Thanks for logging in and the comments.
DeleteNote that the WMC already changed their NY findings so they know that information is incorrect. Its not a matter of using an atypical definition of the years in the cohort.
Also note the discrepancy is arguably almost 50 million cases consumed. That's a lot of wine per person if you are working on the math.
What is needed is for the WMC to repudiate their earlier findings so the digital record is corrected. Then they need to issue corrected and supported findings, then explain what happened so we can trust their research in the future.
You also should be asking why the WMC would use a range that is not in agreement with most research to use your words. Why not try to use a range that is comparable and avoid confusion?
Rob,
ReplyDeleteGreat post. I must admit I was wondering about the discrepancy between WMCs view of the world and yours, which I respect. The idea that they're quietly revising their presentations without any explanations is disturbing.
Alder
Alder -
DeleteThanks for logging in and the comments. If the discussion was my research versus theirs I'd not have posted this. All research has some level of bias but it has to be understood.
The research that I do to come up with Cohort Consumption is to ask wineries who track the information, what percentage of their sales go to each cohort. Because the 'average winery' sells wine at higher price points, I would expect a bias toward non-millennials, simply because I know millennials aren't in their prime earning years. Statistical significance isn't a question with our research as typically we have well in excess of 500 wineries responding representing arguably tens of thousands of consumers.
The only reason I am bringing this up now is the information put out in NY by the WMC was way outside of any other research and I started running into wineries using the information to strategize and using bad information would hurt their chances for success and cost them money. That phenomenon continues to happen to this day, so the effort here is simply to correct the digital record and hopefully prod the WMC to be more transparent about their research findings, process, and support with facts their revised statement about millennials and boomers now being equal in consumption.
We should thank the Board of the WMC and their principals for being willing to take a second look at their findings. That is laudable. What is unfortunate is the conflicting revision to their NY findings was done without a press release or full explanation, so media is still referencing the earlier bad information and the message that millennials are now the dominant cohort is becoming permanently embedded in the public narrative.
If the Wine Market council’s mission is to grow, strengthen, and stabilize the wine market in the U.S. on behalf of all segments of the industry by providing, accurate, fact based information and opinions based on clear, reliable, comparable, repeatable valid data then they should consider the following they should consider the following:
ReplyDelete1. Harmonize as many category definitions as possible with how the beer category (direct wine category competitor) and or soft drinks as possible (i.e. Definition of high frequency user; Budweiser defines heavy user (13% of beer drinkers) as a person who consumes at least 5 beers a week.
About half of Americans drink soft drinks, the average daily amount is a little more than one 12 oz can with 28% drinking one a day and 20% drink more. So a coke heavy user drinks two cans per day or more. Using wine centric industry definitions is confusing, lacks reliability, validity and is subject to interpretation.
2. Recruit at least one neutral non-current wine industry board member who is actually a market research pro from a major CPG (preferably beverage) company. A person with experience interpreting large quantities of consumer research for years and applying it in a multi-brand, multi channel environment to the CPG marketplace. A person who understands the difference between data and information as well as the nuance of how scan data, tracking studies etc are actually constructed. A retired beer or soft drink market research person would make sense.
3. Present all information about volume gains and losses in both units (gallons or 9 liter case equivalents) and dollars together in an easy to understand "frame". The largest wine brand in terms of gallons consumed is called the market leader in beer, water, and soft drinks.
4. Provide the evidence to whether or not the research is valid to a .10, .05. or .01 level of confidence based on the actual numbers.
5. Given the US is quickly becoming a minority majority country, break out ethnicity in relation to any data presented along with the consumption index tor provide clarity. (i.e Hispanic index of wine consumption is 97 vs Anglos 105)
This assumes the mission is as stated above.
APREMAT - thanks for logging in and the comments. I actually haven't read the full WMC mission statement before.
ReplyDelete"grow, strengthen, and stabilize the wine market in the U.S. on behalf of all segments of the industry by providing, accurate, fact based information and opinions based on clear, reliable, comparable, repeatable valid data."
It does seem like they have strayed from their mission by refusing to clarify their findings. We know the original findings aren't accurate and the findings and information aren't clear. I question the reliability of data that can't stand up to validation so, I'm not sure how they are serving their mission or their constituents in the current discussion.
Some interesting thoughts about possible changes to their processes too. Hopefully their board is reading and considers your suggestions in a thoughtful manner.
One thing that would help credibility of all data is published methodology which offers transparency and the possibility of reproducibility and peer review.
ReplyDeleteThis methodology link from the Brewers Association on how they derived craft beer sales data should be an example to follow: https://www.brewersassociation.org/insights/2015-craft-brewing-growth-by-the-numbers/
Lew
DeleteThanks for logging in with a real name. Its always easier to write anything as anonymous or a made up name so I appreciate those willing to offer direct comment.
I totally agree with your comments. In the case of the survey work we do, we publish methodology and one step more, after we clean the responses of obviously bad data and identifying information of the respondents, we turn the complete data set and our analysis over to all the respondents. Anyone could see our questions, how many responses we had to determine statistical significance, verify our research and reproduce what we've done or tell us we erred in the analytics. We've also submitted our data to academia in the hope we could learn something new or develop new analytic approaches.
In the case of WMC information, given the wide divergence from other published information, it would be quite eye opening to see their questions, data set, and try to understand how they arrived at their conclusions. No other research presently done is able to replete their quantitative findings.
Even worse in this case, the WMC released findings in their revision statement (boomers and millennials equal in consumption), and have no quantitative measures to support that foundational statement. Far from asking for the kind of published methodology you suggest to replicate a finding, offering numeric support for a statement would seem to be a minimum requirement for research to have any credibility.
Hi Rob -
ReplyDeletethanks for posting this. The WMC's findings had presented me with some difficulty. As a retailer at the beginning stages of a significant expansion, I'm rethinking our own strategy. I couldn't reconcile the WMC's numbers with our own internal data. I had initially thought that this was the case because we are in an outlying market. While I'm sure this accounts for some disparity between research into broader market trends and our own data/experiences here, there was just too much distance between the two.
Jason
Jason
DeleteThank you for logging in and for the comments.
Having bad information widely disseminated in the business is not only a distraction but it's costly. I don't like being the person pointing this out but it needed to be brought into the light.
For what it's worth the wine market Council issued a press release today "standing by their earlier statements" but at the same time announced that boomers are still the largest consuming cohort. Of course that wasn't the published statement from their Yountville presentation which had Millennials and Boomers "about equal in consumption" but it's movement in the right direction.
I will take the press release issued today as a small victory in trying to get the digital message corrected. While they offer no numeric support for their new conclusion that boomers are the dominant consumer, at least their new finding today is consistent with all of the other credible research available.
"A lie told often enough becomes the truth." ((courtesy of the internet))
ReplyDeleteThanks for the comments Kashy.
DeleteI believe the Wine Market Council plays an important role in the wine business. They have reported findings of fact for many years. Over the past decade however, there have been more alternative sources of information that call some of their past and current conclusions into question.
In order for the Wine Market Council to remain relevant when there are growing sources of credible research giving consistent, but results in conflict with WMC findings, they will need to make some changes and begin a process of becoming more transparent in their methodologies.
To your point above, the entire purpose of this post was to call out bad information and get them to correct the digital record. They have made an attempt in doing that by issuing a press release yesterday where they "stood by their research," yet changed their findings from their Yountville conference on cohort consumption.
In New York Millennials were dominant. By the time they presented in Yountville Boomers and millennials were equal in consumption. Yesterday in their press release Boomers, we were told are still the largest consumer of wine in America.
Each of those statements where made from the same data set - there was no adied research done. In yesterday's press release they reiterated a single error in their New York conclusions. They didn't however present a full accounting of the impact that error had on other conclusions but they did talk through some of it for the first time.
Even more intriguing is their last two changed statements regarding cohort consumption. There were no metrics offered. There were no charts presented which would have supported their findings. They just made the statement and we are supposed to accept that conclusion with no support.
I do believe the board of the organization has very bright people who understand research and fully grasp the incongruous behavior and statements. Credibility is at stake.
So my hope is that they do have the oversight as board members to elicit some significant changes in processes and communication and the WMC continues their mission.
In a scientific paper, there is a "Materials and Methods" section that describes how the data were generated. There are also citations of work done by others so the reader can delve deeper in a way that illuminates the topic at hand. Numerical results may also have statistical tests of significance. The work is usually reviewed by peers before being published. I realize the WMC report is proprietary but the topic is also *really @#$*^ important*. This is not a simple difference of opinion. My money's on Rob.
ReplyDeleteDave Graves
Dave,
ReplyDeleteThanks for the comments and for using your own name. It's unusual for people to want to pin their name to this kind of a conversation. It's not fun. Where is the personal upside engaging in the topic? In my case there are times when you have to do what's right even when it doesn't benefit personally. You are obviously crazy enough to think the same way.
On your other comments, no this isn't a difference of opinion. It's not my opinion against the WMC. It's not my facts against theirs, and there is nothing personal about my comments intended, though I acknowledge they are uncomfortable and poignant for everyone involved including me. No - it's not a difference of opinion at all.
The plain-spoken facts are the WMC released a report in NY with grandiose but error-laden claims that garnered national headlines. The bad information from that report started getting embedded into business plans and conversations in real life and I was constantly put in a place of correcting their mistake when reviewing wine company strategies. They quickly knew the findings were wrong but rather than issuing a press release retracting the findings and moving by the mistake cleanly, they took the unseemly route of quietly removing the headline slides from their presentation deck and in Yountville, issued new and un-sensational findings without retracting the headline from NY. Un-sensational findings don't create headlines and aren't picked up in the press. The result was the digital record didn't change, the bad information was still in the record and being repeated as I noted above with examples. It wasn't until after I wrote this piece that the WMC issued a Press Release which offered some corrections to the narrative.
Is the digital record corrected now? You tell me:
*In NY the WMC said "Millennials were the dominant consumers."
*In Yountville after removing the headline slides, without any figures for support just said, "millennials and boomers were about equal in consumption."
*And in the new press release where they announced they are "sticking by their research," John Gillespie said “The overstatement of glasses per occasion, once corrected, leaves us with the conclusion that while Baby Boomers remain the largest wine consuming generation, Millennials are rapidly closing that gap,” Of course that isn't what was said in Yountville - but they are sticking by their research. They are only changing their findings for a third time and doing it for the second time without issuing any numerical support whatsoever to support their revised findings. It seems rudimentary to me that if you issues a conclusion, you should present some facts, no?
So .... three different statements. An acknowledgement of a mistake, no mention of the headline slides removed (the first two above in this blog) ... So is the record corrected? I don't know but at least now with the blog written and comments from people such as yourself, and 5,000 views so far - there is at least a discussion of the issues faced and a place to go and read about the facts. People can now make up their own minds and decide if the research is worth using in business plans.
Thanks for betting on me. I hope you get a fair return for the wager!
This is interesting to me. I must be missing something. isn't the Wine Market Council and Nielsen related? Isn't the information from this group vetted and shapped with input from Nielsen's information sources? They have been presenting together for a couple years at least so it seems strange the stuff that's being said could be as far off as you say? Like I said, I must be missing something.
ReplyDeleteJerry
DeleteThanks for logging in with a name and the comments
This is a common misperception so let me shed some light on this: The Wine Market Council is in no way owned or associated financially or structurally in any way with Nielsen. As far as their road show goes, having seen the presentation first-hand it's clear to me there is no coordination between the presentations. Nielsen's data is theirs and the WMC information is their own. Their findings do not foot to each other and there is little cross-over in findings.
I know that Danny Brager from Nielsen is on the WMC Research Council, but I don't know if that's honorary or if Danny actually has influence on the analytics. I would guess the answer is the position is honorary as I know him as a buttoned-down professional and if he were involved, I'd have to believe the wild research finding put out in NY would have never seen the light of day.
Further to the point, the most recent Nielsen research I've seen on cohort consumption has millennials below 15% of total share. Since that's not the 42% number WMC threw out, it's pretty clear Nielsen doesn't influence the WMC.