Compared to many engineering disciplines, software development - which began in earnest in the mid-20th Century - is still in its youth. The science and art of software development have progressed in two broad threads: the tools themselves (IDEs, compilers, languages, frameworks, build/CI systems, etc.) and the development methodologies employed - which together determine the efficiency, scalability and flexibility of the development process. At the same time, development processes have evolved from closed, completely internal and highly controlled processes to more collaborative, agile and open approaches, utilizing many open source components.
The methodologies have evolved considerably, as have the tools that enable these methods - from ad hoc to waterfall to agile. Today, we are hearing from customers more and more frequently that they want to gain the benefits of open source community-style collaborative development inside their corporate development organizations – what Tim O’Reilly has called “inner-sourcing.”
Tim O’Reilly coined the term “inner-sourcing” in 2000, describing it as: “the use of open source development techniques within the corporation.” Tim observed even back then that the collaborative, self-motivated, meritocratic process of open source development was different and had several potential advantages over traditional development, particularly on dimensions of improving quality (the multiple eyeballs phenomenon noted by Eric Raymond in “The Cathedral and the Bazaar” in 1999), the ability to enhance innovation (multiple brains collaborating on the same problem), and the sharing and reuse of code. Prof. Dirk Riehle, University Erlangen-Nurnberg, Germany, describes “inner source” in much the same way, and his research on open collaboration within corporations finds similar benefits.
Open source techniques and communities - and the code they produce - have grown and evolved significantly since then, and are still accelerating. Consider that the number of unique open source projects will exceed 600,000 this year, has been growing 35-40% CAGR for the last six years - and more than doubled in the last two alone.
“Outsourcing” began in the 1980s and accelerated in the 1990s as development organizations worked through an early wave of cost cutting and efficiency initiatives. Competition forced development teams to prioritize and focus on what they did well and where they could deliver premium value in the software they created. Development of non-core, low value features/functions were “outsourced” to the cheapest bidder, and outsourcing helped deliver more value from increasingly scarce internal development resources.
Inner-sourcing is driven by similar but more evolved motivations.
Corporate IT has made extensive use of open source code for years. Gartner reported that on average, 29% of deployed code was open source, and that by 2015 at least 95% of mainstream IT organizations will leverage open source solutions within mission critical software deployments. So while open source code is being widely adopted, it’s only recently that corporate IT became interested in the efficiencies of the open, collaborative creation process itself. Projects spin up quickly and attract contributors organically without advertising or hiring; large distributed teams produce high quality innovative code with little overhead; and it’s all done completely in the open.
Open collaborative development via communities is widely understood and accepted, and corporate IT organizations are realizing that these characteristics can be applied to improve their internal development as well, and many are looking to apply them to enhance their own internal methods, typically in conjunction with adopting agile or lean methodologies. Given the inbound interest into Black Duck’s Olliance Group on this topic, in my view, this will soon become an influential method within development organizations.
The benefits these organizations are consistently looking for, when exploring inner-sourcing, include:
- Code reuse
- Better quality
- Improved innovation
- Cross-organization visibility into code, projects, skillsets
- Cross-organization collaboration, buy-in
- Developer engagement and morale, motivation, volunteerism
Let’s take a look at the characteristics of open source methods that will enable and empower inner-sourcing:Transparent and Collaborative
In a corporate IT environment, transparency and collaboration can enable people with expert resources to contribute and provide feedback where previously it was not possible, it can compel and attract engagement, and attract new community members, perhaps contributing in their spare time, i.e., corporate developers paradoxically “moonlighting” or volunteering for other projects within their own company.
Developers decide to join (or not) and contribute to projects based on a number of factors including how interesting the work is, the ability to make an impact and be recognized for it, whether the person considering engaging needs the software or finds it valuable. The more transparent the process, the more information will be available, and the better that information will be for potential contributors to act or respond.
Corporate IT developers share many of the same motivational self-interests of open source community developers around solving problems and being recognized for their contributions. Providing higher levels of visibility and information in a systematic way enables developers to volunteer and self-organize around areas of interest and/or their unique skills and capabilities. In the corporate IT case, the organization must sponsor and endorse the process, but not dictate. In the best-practice cases Black Duck has worked with, there is often a cross-department, self-formed and managed steering group that provides the leadership and “activation energy” that gets the community-style collaboration going.
The open source community values and provides public feedback, both positive and negative, on contributions and contributors. While some open source communities exhibit somewhat harsh personal criticisms, many participants value its apolitical nature, for if you can succeed in a difficult environment, you can burnish your reputation.
Recognition of one’s contributions can be in the form of individual comments and feedback. Some communities have created an infrastructure for the community to provide recognition of achievements. Jono Bacon, the Ubuntu Community Manager from Canonical, recently wrote an excellent blog about their approach for recognizing achievements: “The Gamification of Community.” In a corporate environment, peer feedback and recognition can be a new and powerful form of motivation, while “gamification” techniques can provide more formal recognition tailored to corporate objectives.
While I’ve been describing inner-sourcing methods, it’s important to note that development tools are evolving in this area as well, and enabling more social aspects of development. The source code management (SCM) project Git first took hold in the open source community as its dynamic forking and merging capability was well-suited to the distributed and independent nature of the open process. Git is becoming widely adopted in corporate IT environments indicating the willingness and interest of developers to exploit the same non-linear development benefits. Git’s popularity in corporate IT is an example of how developer self-interests can be a powerful positive force.
Another enhancement to the development tool set is likely to include a way to capture actionable metadata on the code, contributions and contributors, and make it available to the internal community. Making metadata available, such as performance attributes and behavioral data (coding trends, comments, commits per developer, etc.) provide insights which can foster collaboration and suggest areas for process innovation. We’ve seen this work on Open Hub, our free community resource, and expect corporate IT would benefit from it as well.
It’s no secret that corporate IT seeks to increase efficiency, scalability and quality. Inner-sourcing can provide a new approach to achieving these benefits, and it has already been proven – in thousands of successful open source project communities. And while many will observe that community-driven open source processes have been outpacing standard corporate IT processes for years, the level of interest I see on the part of corporate IT suggests that is about to change.