Sunday, July 5, 2015

250,000 Credit Cards Stolen in Wine Industry Hack

ECommerce Payments
I've always thought the wine industry should be the most uninteresting industry for cybercrime. Wineries have lots of inventory to steal, but anyone who works in this business knows there isn't much cash to take. It all gets used up in barrels, bottles, inventory, and facilities.

Who and even more to the point, why would anyone bother to hack into a winery? It's not like there are any huge IP secrets to take. North Korea doesn't care about the 2015 vintage. Chinese spies have to get paid more to focus on our Government's and defense contractor's systems rather than messing with wineries I'd think. Pre-pubescent teenagers trying to hack winery computer systems would have more fun trying to hack celebrities' personal sites or play World of Warcraft. That's where young people can really experience virtual power and control.

Besides, the wine business is really a bitty industry; one full of mom and pop shops. So why would anyone bother to try and hack into a winery when there seem to be so many other far more interesting and larger industry targets out there to probe?

That question is no longer academic because today - right this second, hundreds of people in probably 100 separate companies are cleaning up after the personal information of 250,000 winery customers was hacked in a recent data breach. [i]

This is a really big deal. While I've heard no mention of the cost of this, it has to easily be millions of dollars in the aggregate given the number of businesses and impacted people who are cleaning up the mess thus far. And those losses are before considering any fraudulent credit card purchases which may have happened or may still happen.

     Risks of Cyber-Threats Growing
Banks have a lot of security measures for obvious reasons so I need to be careful in describing them here, but suffice it to say that banks have to monitor client computers that link to the bank's online banking platform to protect against an attack from that angle. That is the weakest link and it's the easiest place for a cyber-attack on a bank.
Over the past few years, I've noticed increasing attempts at winery computer hacks, and a higher degree of client computers infected with malicious code without the client's knowledge.

When a client's computer has something malicious embedded, we have to freeze their online account until they repair the problem. They are understandably frustrated by having their online banking disconnected versus thankful we found a problem that escaped their IT security measures. 
But I think that perspective really speaks more to the fact most people just don't see the consequence of this cyber threat. In their mind, their Norton Anti-Virus and Microsoft Firewall are sufficient and the bank is being unduly forceful by shutting down their electronic banking access. I understand it's frustrating and feels like poor client care, but it's the exact opposite.
Cyber attacks are getting worse by the day. Hackers are getting more sophisticated and organized by the minute and that's why I think this is such an important blog to write. The wine business has to get hold of this new and evolving reality and take immediate action to protect their businesses.
Even if you have not mentally linked this threat to the wine business in the past .... maybe it just never occurred to you yet, I'll bet you've nevertheless noticed the incidents of successful breaches mentioned in the press are increasing at a quicker pace:

      Wake Up Call: The Wine Industry Gets Hacked

Those are all really big stories and unimaginable costs to bear but until now I'd not previously heard of successful winery data breaches. Well... there was a small story recently about ISIS hacking a winery network in Washington recently for propaganda purposes, but that was a road bump.

It's now been made public in several publications but specifically this one [LINK] that a small e-commerce company called eCellars had their winery customers' client information stolen in April.

Let me repeat that so you track it: The information that was taken was hacked from eCellars computers and not the wineries. What was taken was the personal information of the winery customers themselves, like wine club members. It wasn't the wineries that were hacked, but their clients' personal information was stolen from the vendor who they trusted with their club members' credit cards and other personal information.

The breach is estimated by one person [LINK] to have compromised the credit card and personal information of 250,000 wine lovers. In the words of a GM from one of the wineries who is not one of my clients,
"I had no idea of the risks involved here. I've had to start by educating myself before I could do anything but I keep needing to react to something before I understand it. This has turned into a nightmare of unimaginable and seemingly unending proportions. Every time I think I have the risks plugged, I find a new one."
I would add none of the wineries dealing with this are happy as you might imagine. It doesn't seem fair in their view. They are now having to bear the consequences and costs of what they perceive as the mistake of a vendor. While that's partially true, it's the responsibility of the winery to know their vendors are PCI compliant. (More on that below.)

      Motive, Opportunity, and Intent

My earlier thinking about this business threat was entirely naive. The reason hackers want to get into your systems is winery customers are wealthy individuals and their personal information, in the hand of a skilled cybercriminal could allow them access to stocks, pension accounts, sensitive personal data that could be leveraged, credit cards numbers, Amazon logons, health records, emails and even paid subscriptions to NetFlix and Consumers Report.

Wineries it turns out are a really good target because your customers are in the top 10% of wage earners and your security systems are easier to hack than larger organizations. While there might not be a lot of IP to steal in this business, your customer's information can be used in countless ways that could hurt both them and your winery.

      Cost of a Breach

Experian estimates the average cost of a hack in 2015 will be $3,500,000 so it won't surprise me if this one costs the industry seven figures and that ignores the costs of any fraudulent use of credit cards by the thief.

Here is a plausible scenario that hasn't taken place as far as I am aware, but is instructive as to the potential cost to a winery from just one vector. In a phishing attack, a criminal could send out an email to your club under your winery letterhead and make a fake offer for your wine like this:

Dear Club Member,
We found 4 cases of the acclaimed 20XX 100 point wine remaining in the cellar. The wine sells for $350 on the secondary market but we are offering it to only a few of our best club members at the $175 release price. Checks only. First come first served. Sincerely - Chapter 7 Wine Company.

What would that cost this hypothetical winery? If there were 4,000 people in the club and 40% responded to the too-good-to-be-true offer asking for the four cases, the thieves would 'earn' $3,360,000 from your customers - each of whom will come to you and ask for their money back.

Since I know the average winery gets about 60% of its income from direct sources, Club members are critical to the success of small wineries. So in this hypothetical circumstance, you would scramble to find ways to accommodate the club members. But can you do that? This kind of mistake could easily put you out of business with the fraud alone. And when it comes to credit card fraud, there are added costs to consider including potential fines and litigation.

      PCI Fines & Litigation

Retailers are required to follow guidelines established by the Payment Card Industry ("PCI") and be compliant with the regulations, one of which is you are responsible for knowing your vendors are PCI compliant. In many cases depending on the level assigned by the card issuers themselves, you may be required to ensure your vendors are continuously compliant. That means you need to get quarterly reports from your e-commerce vendor of their current certification.

Fines can be issued from the card issuers like Visa, Discovery, and the like. The costs of cleanup and any card losses can be passed down to the wineries in this case, even though the hack wasn't on their systems.

Separate from fines and costs getting passed back, this is one of the fastest-growing parts of the legal system and litigators are organizing into a new industry servicing impacted persons and companies. Massive lawsuits have already been filed due to the large public hacks described above, so getting this right is something for which anyone who takes credit card payments should take notice and that obviously includes the wine business.

      How Vulnerable is the Wine Industry?, the answer to the subtitle question is "extremely vulnerable" and I have facts to support that.

PCI Compliance is a shorthand way of saying your business is keeping current with security protocols and your immediate and extended network is secure. If you process credit cards, you are now, or will soon be required to provide your issuing bank with proof of compliance.

That's a little concerning to me because when we did a survey last year asking wineries if they were PCI compliant, two-thirds said they weren't compliant and nearly a third didn't even know what PCI Compliance meant!

      What To Do When You Are Hacked?

It has to start with prevention and preparation. After talking to some of the wineries who are still cleaning this up, they tell me the most important things you should do are:

  1. Put an employee in charge of IT Security and PCI Compliance and then educate yourself so you know how to oversee the business function. Start by making sure your systems are secure and check that quarterly for needed changes.
  2. Have a dedicated computer for banking that never is exposed to other internet sites.
  3. Have a Contingency Plan and Business Response Team at the ready. Include in that plan all the resources you may want to use including the names of attorneys, and PR professionals, your financial institution, and local FBI contacts.
  4. Review your Business Risks Insurance Policy to make sure you are properly covered for cybercrime. (Some cheap policies say you are covered but really you have little coverage at all.)
  5. Run scenarios around a conference room table with the response team so you aren't starting from scratch when you are hacked.
                     eCellars Actions

The owner of eCellars probably has some practical experience to offer and I hope he does when he can see things clear to do so. I can't begin to imagine how he felt when he started to come to grips with this but it's probably instructive to put yourself in his shoes and consider what he's had to do in a very short time. 

I'm presuming early on the FBI was involved either because they notified him of the breach or he discovered it and had to notify the FBI. Either way, he had to first discover exactly what information was taken, where the security breach took place, and repair the security architecture.

I presume that he had to take the site down for a period while he was going through this discovery which couldn't have made his winery customers happy and probably cost him revenue. Undoubtedly he needed to have an attorney brought in with specialized skills. Then after all that, he had to open Pandora's box and tell his clients of the incident. Explaining the situation to his client wineries, here is a small part of a letter he sent which has since been published:

"The intruder gained access to customer names, credit/debit card numbers, the related billing addresses, and any dates of birth in our system during the window of April 1st through 30th this year. The intruder did not have access to any driver license numbers, Social Security numbers, CVV verification numbers, or PIN numbers (data which we would typically not collect anyway). We have identified and secured the method that was used to breach our platform."
Another action eCellars has taken to repair the breach and be compliant is to outsource the protection of the winery customers' personal information to a single vendor (OpenEdge). In this case, eCellars was able to have this vendor handle the security but had to promise the winery's e-commerce business as part of the package. The wineries aren't given a choice. If they want to remain with eCellars, they have to move their card processing to OpenEdge. It's one more issue the wineries have to deal with but I'm guessing they will all take this path largely because of the difficulty in going through an RFP process at this time, as well as the added difficulty and learning curve needed when changing an ERP system.

I think some of these details are worth hearing because cleaning up these kinds of problems is far more difficult than most would believe.

                    Other Steps To Consider

This is far too complex to cover in full, but here are other suggestions to consider adding to your Contingency Plan:
  1. Define what could be taken, what your response should be, and who needs to know.
  2. The first 48 hours are the most important. The response team should know what their roles are and who they need to inform of findings and progress. Don't presume the threat plugged is the only one. Since you've been attacked, someone knows where you live. Make sure there is no secondary continuing active threat.
  3. The second phase of clean-up has to get the business back up and running. What should you say to constituents: employees, board of directors, public, law enforcement, financial partners, and regulators.
  4. Include a feedback loop. Talk about the situation openly and include new learnings and discoveries. Update the Contingency Plan.

Additional Resources:

[i] Footnote:  I've not seen the source of the 250,000 customer figure cited by the blogger linked to the story, but it's easy to get there even as an estimate: If eCellars has only 70 clients, and the average wine club is 4,000 people that is already 280,000 individual records exposed.
The number of businesses involved in the cleanup is an estimate but probably a conservative one.  Consider the estimated 70 wineries of eCellars, then add in all the IT companies helping the wineries, as well as law firms who have been retained to send out the notification letters to the state's Attorney Generals. Banks are involved too. Interestingly while Silicon Valley Bank wasn't involved directly in this beach, some of our clients and prospects have come to us for guidance, so we are very involved.

What Do You Think?

Please join the site in the upper right hand of the page and offer your own thoughts for the benefit of the wine community.  If you think the discussion is worthwhile, please promote this on your favorite social media platform.



  1. Shaun M. McDonaldJuly 7, 2015 at 1:14 AM

    Pretty sure CA will find a Tied-House violation here to profit by (sigh)

  2. Shaun - Tied-house isn't related to the Payment Card Industry and security so I don't think that's the case.

  3. Rob - To clarify the company name is eCellar.