Before joining my first software company, I spent several years with Whistler Radar Detectors. We manufactured and sold over half million little radar detectors every year. As you can imagine, keeping track of parts from dozens of suppliers was a big deal. We used this information to place orders for new parts, of course. We also used it to standardize our parts. If we could use the same capacitor, resistor, or screw across multiple products, we could order larger quantities to reduce costs and gain leverage with suppliers.
Equally important was tracking the quality of each component. A bad component could cost the company a lot of money in returns, warranty expenses, and rework (have you ever tried reflowing solder on a surface-mount component?). If a problem arose in a part, resulting in early product failure, we wanted to know which individual units were affected by tracking inventory and product serial numbers. If we didn’t have that parts list (bill of materials) for each product, we would have been lost.
Twenty years ago, a software parts list would have seemed ludicrous. All software was built from scratch, and every code base was unique. Today, however, a software bill of materials is critical to organizations, for many of the same reasons they are required in radar detectors.
Today, Software Has “Parts”
Open source software has fundamentally changed how software is “built.” Our research shows that even in commercial software, open source comprises over one third of the average code base. In organizations that build and consume their own software, it’s not unusual to see applications that are comprised of 80% or more open source. Understanding what “parts” your software teams use in each application is a requirement if you intend to comply with copyright laws or defend your applications against attacks.
Just as we worked to minimize the number of unique screws, waveguides, and LEDs in our radar detectors, organizations want to minimize the variables across their applications. We’ve seen dozens of versions of the same component in a single application. We’ve also seen different open source components that provide identical functionality in the same application. Standardizing on the open source used within an organization simplifies maintainability and makes tracking issues easier. If this is preceded by a thorough review, it will also ensure that you and your software team are selecting components that have strong community support and a good record on security.
With radar detectors, we worried about component failure and classified that as “quality.” With software, component failure is a security issue. Open source is software, written by humans. It will have weaknesses that can result in vulnerabilities, the same as commercial software.
The difference with open source is that there is no vendor “pushing” you updates and security bulletins. It’s up to you, the open source user, to track these issues and “pull” the updates into your codebase. How do you do this without knowing exactly which open source “parts” you’re using? With over 3,000 new vulnerabilities disclosed each year, having visibility to your open source is critical.
Avoiding the (Keystone) Cops
I know, radar detectors can help you avoid tickets, but I’m thinking more about incident response. Think back to when Heartbleed was disclosed in April 2014. In most organizations, this was a Keystone Cops moment; people running around trying to figure out where they used OpenSSL (because everyone was using it someplace). Most companies loaded up a vulnerability scanner with the Heartbleed rulepacks and started scanning everything in their environment. It took days and weeks to find the problem applications and systems, and we’re still seeing vulnerable applications today (almost 1.5% of the on-demand audits we conducted in 2016 included Heartbleed — read the report).
This would not have happened in our radar detector company, simply because we had a bill of materials and knew precisely which parts were in which products. Nor does it happen when an automotive company has a recall, because they maintain a bill of materials for every car.
It also didn’t happen with Black Duck customers, because they, too, had a bill of materials for every application. Instead, when Heartbleed was disclosed, they looked up OpenSSL and could immediately see every version in their application inventory, and which applications used each. With that information, they could then point their vulnerability scanner at those specific applications to determine exploitability and wrap up their triage process.
Look, Listen, and Learn
As an industry, we need to be willing to learn from other industries. This isn’t necessarily a natural trait for security people, but the problem of defective parts was solved over 100 years ago by the auto industry and other manufacturers. An accurate bill of materials makes life a lot simpler, and your products more reliable.