I joined Black Duck in 2004 after having worked for over a decade for other IT consulting companies and software companies. The company I joined was very different from the company that employs me today. In 2004, Black Duck, though funded and shipping products, was essentially a startup in both culture and capabilities. Today, we are a mature company with proficiencies and resources far beyond a decade ago, however, our core focus on open source license and version information remains.
Increasing Maturity in Managing Open Source
Our transition and growth at Black Duck mirrors the changes and increased maturity of the best practices in managing and securing open source for an enterprise. The primary lesson we have learned over the years is that open source management needs to be agile, help simplify license compliance and provide strong support for the management of open source security risks. So, how did we get there?
Fear of (Open Source) License Infringement
From 2004 into late 2007 the big news revolved around the SCO lawsuits, Linux and the formulation of the GPLv3. The state of open source management was primarily focused on fear of license infringement, particularly on managing the use of code under reciprocal licenses like the GPL. Though there was a large quantity of open source, compared to today open source had limited availability and use. Managing open source was viewed as only a problem for those who redistributed the open source (typically tech companies). Best practices to identify open source relied on using scanning tools to identify reuse in source code. The focus was on finding individual files and even fragments of code. Many (primarily tech) companies invested in tools and programs to audit code to discover and manage those risks, and Black Duck was there to help with products and services.
A Shifting Focus
Starting in late 2007, the focus began to shift and scanning for open source was not enough. Now, the first lawsuits (that I can recall) were filed around the GPL, and those lawsuits prompted more focus on general open source governance. The big news at the time were those lawsuits, particularly the BusyBox lawsuits and the actions of open source advocacy groups like the Software Freedom Law Center and gpl-violations.org to protect the rights of open source developers and users.
Though the lawsuits were typically settled, generally they resulted in the defendant coming into compliance with the licenses, paying some type of financial penalty and (to me this is an interesting item) appointing an open source compliance officer. As such, companies began to focus on their open source governance programs, including policies and processes. In addition, the introduction of licenses like the Affero GPL, which can cause obligations to come into effect when the software is used over a network (like the internet), meant that companies who did not distribute technology but did provide SaaS applications and other software services needed to pay attention to their open source use and the attendant licenses.
Best Practices at the Time
During that period, best practices involved creating an approval process to manage the introduction of open source, maintaining a database with that information and implementing pre-release/pre-deploy steps in the company’s software delivery processes to ensure compliance (which included scanning). Often, the approval process required multiple gates and approvals. While company’s management was introducing more governance, the adoption of open source was significantly increasing in their software development organizations. Therefore, though everything needed to be approved, the processes had to be automated as much as possible. It was at this point that non-tech companies (such as enterprise IT organizations) were becoming more aware of the need to keep accurate inventories of open source in use, especially to manage any security vulnerability risks.
Open Source is Everywhere
Fast forward to today. In some ways, things are more complex. Open source is everywhere and mission-critical to both tech companies and regular enterprises. The number of open source components in a typical application has exploded. The is primarily due to the rise of number of contributors and the adoption of modern package management systems that help companies manage (from a software development aspect) the complexities of large numbers of components.
Today, open source tends to be incorporated at the package level, and once the dependencies of those packages are resolved, 30 components you want to use become 1,000 that you must use. In addition, the adoption of containers, micro services, continuous deployment and other factors have exploded the number of “releases” produced by software development groups. This growing number of releases and open source components in use is not only making license issues more complex, but can be a nightmare for security organizations. Certainly, a lot of the big news around open source has been regarding security vulnerabilities, starting with Heartbleed in 2014.
Open Source Management Today
So, what can companies do today to manage open source? Many of the processes developed a decade ago do not scale in today’s software development environment. Today’s processes need to be agile, automated and as simple as possible. Best practices include using tools to create automated, accurate inventories of open source with minimal manual review. Open source is introduced quickly, and reviewing and approving all components won’t scale. Exception based policies that can identify unacceptable open source across multiple factors (like licensing, security or other risk factors) for additional review are the only ones that work without introducing bottlenecks. These tools and processes need to be integrated into the DevOps processes, likely at multiple points. They need to take advantage of information from package management systems and other factors. Fast.. Simple…. Accurate.. Integrated… As always, Black Duck is here to help.