As most of you are aware, last Friday news broke of a major data breach at Equifax. As one of the major credit reporting agencies, Equifax maintains a vast amount of sensitive personal and financial information for residents of the United States and the United Kingdom, and this breach is reported to have compromised the information for nearly 150 million US and UK citizens.
Equifax has not issued a public statement pinpointing which vulnerability was exploited. However, multiple sources have reported it was a vulnerability in Apache Struts, a free, open source framework for creating web applications. It is worth noting that Apache Struts is widely used by Fortune 100 companies to build corporate websites in sectors including education, government, financial services, retail and media.
What Do We Know About the Apache Struts Vulnerability?
The breach is being attributed to exploit of a vulnerability in the open source Apache Struts framework. Accordingly, the Apache Struts Project Management Committee released a statement regarding the Equifax breach that includes excellent suggestions for securing any open or closed source supporting libraries in software products and services, which I'll share verbatim (emphasis mine):
- Understand which supporting frameworks and libraries are used in your software products and in which versions. Keep track of security announcements affecting this products and versions.
- Establish a process to quickly roll out a security fix release of your software product once supporting frameworks or libraries needs to be updated for security reasons. Best is to think in terms of hours or a few days, not weeks or months. Most breaches we become aware of are caused by failure to update software components that are known to be vulnerable for months or even years.
- Any complex software contains flaws. Don't build your security policy on the assumption that supporting software products are flawless, especially in terms of security vulnerabilities.
- Establish security layers. It is good software engineering practice to have individually secured layers behind a public-facing presentation layer such as the Apache Struts framework. A breach into the presentation layer should never empower access to significant or even all back-end information resources.
- Establish monitoring for unusual access patterns to your public Web resources. Nowadays there are a lot of open source and commercial products available to detect such patterns and give alerts. We recommend such monitoring as good operations practice for business critical Web-based services.
These recommendations provide a great baseline for preventing breaches like the one Equifax just disclosed.
Ok, Which Vulnerability?
While early speculation suggested that the vulnerability was CVE-2017-9805, which was reported at roughly the same time as the Equifax announcement, many now believe the more likely culprit is CVE-2017-5638, which was reported back in March of this year. At the time the time CVE-2017-5638 was reported there were already exploits available and several sites were reporting seeing active attempts to scan and use them.
Equifax states that the data breaches spanned from mid-May through July. Our Open Source Security & Risk Analysis (OSSRA) suggests that it's likely Equifax does not have a complete view of the open source in their code. We found that companies were using twice as much open source as they thought; 67% of analyzed applications using open source had vulnerabilities in the components used; and on average, vulnerabilities identified in these applications had been publicly known for over four years. So while the Equifax security team may have been aware of this vulnerability, the applications that were hacked may have flown completely under the radar. This would explain how an open source vulnerability disclosed in March could still be used by hackers during the May-July timeframe.
What's interesting is that a patch for CVE-2017-5638 was available back in March when the vulnerability was disclosed... well before the May timeframe for the Equifax breach. This ArsTechnica article provides a possible explanation for the delay in applying the patch:
"Ars Technology Editor Peter Bright provides a much more plausible explanation for the delay in patching this highly critical vulnerability. Most bug fixes, he pointed out, require downloading and installing a patch, possibly rebooting a machine, and being done with it. The fix here, by contrast, typically requires each Web app that was developed with a vulnerable version of Apache Struts to be recompiled using a patched version."
So why would Equifax have left the vuln in place for another two months? It could be that the team at Equifax was working on applying that patch the whole time and just didn't get it rolled out soon enough. But given the severity of the vulnerability, it seems unlikely that they would have knowingly left the hole open that long. The more likely answer seems to be that nobody was tracking this particular piece of software.
How Lack of Visibility May Have Opened the Door to Exploit
Like Heartbleed, this incident shines a light on the issue of open source security and the race between you and would-be open source hackers once a vulnerability is reported. Because open source is so widely used and easy to access, it's very common for exploits to be available on YouTube and hacker sites immediately after a vulnerability is first reported. Those exploits are like a set of keys hackers can use to break into thousands of websites and applications. This is the time when you are at the greatest risk of breach.
In this case, Equifax, like many companies, has a large portfolio of applications. As revealed in our OSSRA report, most companies aren't doing a good job at tracking open source, so unless Equifax had deployed a solution like Black Duck Hub, they probably did not have a complete and reliable inventory of the open source components in use in their applications. So in March, when the vulnerability was disclosed, it would be highly likely that they would not even know they were at risk, even if their security team was aware of the vulnerability. Put simply, they were flying blind.
Since the exploits for CVE-2017-5638 were widely available and being used almost immediately after the vulnerability was disclosed, Equifax entered this period of very high risk without knowing it, at the same time that hackers were actively scanning and probing to find websites and applications that were vulnerable. If this is the case, the door was "unlocked" until they discovered the breach over four months later.
Whatever regulatory rules apply to the specific app probably required them to confirm if any data was accessed, which is when they brought in the forensics team. Given the 2017 Cost of Data Breach Report from the Ponemon Institute stated an average of 206 days to identify and contain breach, this timeline shows Equifax is actually above average in their response time.
So What Can Companies Learn From This?
Aside from the obvious privacy, litigation, and even political fall-out from this breach, executives and security teams may be wondering if they too are at risk from this or other open source vulnerabilities? In the end, there are many lessons that can be learned that will help teams avoid this type of security lapse.
- Visibility is critical. You can't protect yourself if you don't know what's in your code. If you don't have a complete inventory of the open source your teams are using then you are leaving your applications at risk.
- Open Source Vulnerability Management needs to be automated and tightly integrated into development and DevOps tools and processes. You are only as secure as your weakest link. Only by ensuring that all code is scanned before going into production can you be confident that you have addressed the weak links.
- Do everything you can do to minimize the window between when vulnerabilities are reported and when you patch or mitigate them. More than 10 new open source vulnerabilities are reported every day. Unfortunately, you can't rely on the National Vulnerabilities Database (NVD) to give you early warning of them. Exploits are already available for the latest Struts vuln (CVE-2017-9805), yet NVD still has no data for it. Our research has shown that it takes an average of three weeks for vulnerabilities to be documented in NVD. To solve this problem, Black Duck independently monitors and researches vulnerabilities using hundreds of sources so we can provide same day alerts for vulnerabilities like CVE-2017-9805.
Unfortunately, when high profile breaches like this happen, it can be tempting for people to throw the baby out with the bathwater by suggesting that open source software itself is less secure. We believe that open source software is no more or less secure than any other software. However, like any software, if you don't track it and stay current with security updates, you are leaving yourself and your customers' systems and data at risk.