In early 2015, Gartner predicted 2016 to be the year that DevOps goes mainstream, being adopted by 25% of Global 2000 companies. As we begin 2017, DevOps adoption is becoming the new norm as the benefits are realised by a wider audience.
Another key priority for organisations is IT security. With 64,000 incidents and 2,300 breaches in 2016 alone, it’s easy to see why the protection of personal data has become increasingly important to businesses and individuals alike.
Gartner recently reported that for 90% of companies using DevOps, security is an afterthought. By 2019, a predicted 70% of enterprise DevOps initiatives will have realised the importance of incorporating security into the foundations of their DevOps practices. Coined by analyst Neil MacDonald in 2012, Gartner calls this DevSecOps.
“All too often, we tack on security testing at the end of the delivery process. This typically means we discover significant problems, that are very expensive and painful to fix once development is complete, and which could have been avoided altogether if security experts had worked with delivery teams throughout the delivery process.”
Puppet State of DevOps Report, 2016
As DevOps becomes the “new normal,” and as security becomes an ever-important part of modern business, teams must build security into DevOps practices.
To remain true to the spirit of DevOps, security needs to be built in at the beginning of the delivery process – at ground zero – and embrace the philosophy of teamwork, coordination, agility and shared responsibility.
In this article, consultants from ECS Digital and ECS Security practices explore why it’s so important to build security into your DevOps practices, how to facilitate this relationship, and how far we are from Gartner’s DevSecOps.
“DevOps teams are delivering at a velocity that security teams are simply not structured to keep up with. By owning the security problem, DevOps teams are more self-sufficient and able to deliver rugged products at speed.”
Jeremy Foote, Managing Consultant at ECS Security
Why do organisations need to build security into the foundations of DevOps?
Put simply, to save time and money by preventing security incidents.
If security is integrated into the foundations of DevOps, teams can feedback on and deal with security issues as they arise, instead of at the end of a lifecycle. Typically, we see that a lot of applications at large enterprises have a final “security check” which often takes weeks, in some cases, months to complete. This slows down the whole process and is a blocker to any DevOps initiative.
By shortening the feedback loop between doing and passing security checks, teams can decrease the number of issues later down the line and improve the security of their applications and environments.
“High [DevOps] performers spend 50 percent less time remediating security issues than low performers.”
Puppet State of DevOps Report 2016
How do you build security into the foundations of your DevOps?
1. Understand what’s at stake if you don’t get security right
A good starting point for DevOps and security teams is understanding the risk of getting security wrong.
Make sure you know the answers to the following questions:
- Do I know what would happen if my applications were to unexpectedly fail or were breached?
- What about the impacts or pain that would be caused to my organisation from a security breach?
- Finally, is my company equipped to deal with these risks, or do we need to invest?
It’s important that organisations understand the worst-case scenario should security be compromised – whether it’s financial or reputational damage.
“The average cost of a breach to large businesses was £36,500, while the biggest cost of a breach recorded in the survey as a whole was £3 million.”
Business Matters Magazine, 2016
2. Focus your security efforts where they matter most
Understanding the worst-case scenario of a security breach helps teams to recognise which applications or systems they should be focusing resources on. Organisations can then make the most of their resources.
Your teams should be aware of which technical components might be exposed, should security be compromised, as well as potential motives for individuals to breach your applications. If a team does not understand the security implications or why it is needed, they will just ignore it.
Teams can then ensure that systems are as secure as they can be, and that testing occurs at the right places.
A vital component of focusing security efforts is communication with the rest of the team, something encouraged through the DevOps methodology. This helps to make sure all teams within IT, Security and business are rowing in the same direction, and delivering with speed, quality and consistency.
3. Provide freedom (but monitor everything!)
Building security effectively into DevOps requires teams to have the freedom to operate freely, and benefit from shared responsibilities.
Freedom enables teams to work effectively whilst shared responsibility means that teams can work quickly towards recovering security issues, without the fear of blame.
“The hallmarks of a generative organization are good information flow, high cooperation and trust, bridging between teams, and conscious inquiry.”
Puppet State of DevOps Report 2016
At the same time as providing freedom, businesses need to monitor and manage identity and access throughout their systems and applications. Everyone in the organisation should have access and permissions only to the areas they need. This way, businesses can rest assured that the applications that need to remain secure, are.
4. Automate and continuously assess your vulnerabilities
Automation is a key element of DevOps. It supports rapid change within businesses, in a controlled and compliant fashion, which enables them to work at pace.
Your goal should be to include security testing into the daily work of Dev, QA, infrastructure and Operations and automate as much as possible. This alone will go a long way to ensure security issues aren’t tackled at the end of the delivery process. Every manual process could potentially be a security risk, or introduce security debt which is more costly to address over time. However, it’s important to make sure that your software development is secure and identity protected, before you can automate it (and remain confident in the security of your business).
End-to-end automation during development, testing and operations means teams can generate evidence on demand to demonstrate that controls are operating effectively. Such evidence is a requirement for auditors and assessors, and beneficial for anyone else working in our value stream.
“The automation of security processes in DevOps enables teams to discover significant problems — including architectural flaws — that are very expensive and painful to fix once development is complete.”
Puppet State of DevOps Report 2016.
5. Protect your toolchain and your code
Whilst there are many amazing tools available on the market, effective processes and implementation are key. A well-defined and optimised process supported by an average tool will deliver a better outcome than a poor process supported by an amazing tool. Processes support people and are supported by tools. People, Process, Tools is a deliberate order that defines importance and where to focus effort.
At ECS Digital, we call this “People, Processes, Tools”.
Your toolchain controls everything; it provides a backdoor into all applications and infrastructure. If it’s not protected, you risk losing control of your product, which can have serious implications. For example: provisioning infrastructure dynamically for a purpose and tearing down afterwards provides great assurance that the environment will be the desired configuration.
If security concerns have not been addressed in the toolchain (who has access to what?) then it may allow inappropriate access, which may cause unauthorised changes leading, to vulnerabilities in your product.
What are the main obstacles when building Security into DevOps?
Whilst bringing security into DevOps can enhance a company’s ability to innovate and prevent security becoming a roadblock, there are always a few obstacles to adopting new ways of working in an organisation. Some of these include:Differing priorities
Security teams are not traditionally included as a DevOps stakeholder, which means they can have different priorities. To build security effectively into the foundations of DevOps, they must be included and their priorities aligned.Going at pace
Deploying at pace using DevOps can be viewed as risky by security teams as mistakes might be made. Aim to maximise the use of automation as part of your DevOps adoption, minimising human errors as much as possible whilst moving at speed.Adopting DevOps incorrectly
There are many horror stories around companies adopting DevOps incorrectly, and most of these failures are to do with culture. If businesses fail to embrace DevOps across entire organisations, it is often only enabled in certain sections. This can then cause further silos, resulting in failure.
In other words, if you have the wrong culture, your DevOps and your DevSecOps projects are likely to fail.Dealing with legacy systems and teams
There are battles adopting different cultures, processes and tooling within any organisation. But these can be more prominent within a traditional organisation, where people and processes have been working in certain ways for some time. We believe that all organisations – big, small, old or new – can adopt elements of DevOps to help achieve their business objectives.Maintaining governance structure
Working as a unified and aligned team (physical or virtual) removes security from the sole hands of security professionals. This change in governance could create issues between teams used to being in control, and has the potential to create issues for other teams without the correct training or background. The management of security governance needs to remain a key priority of businesses to get this right.
Know when to get help!
Whilst ECS Digital fully believes in the results of DevOps, like any framework or collection of methodologies, regular feedback is key to ensuring that you’re delivering products securely and to also look at how to improve from past situations. Companies must remember the significance of knowing the status of and gaps in their workflows, always.
Only through complete transparency can teams know when they’re working securely, and when they might need additional help.
Soon, many organisations will have adopted DevOps in one way or another.
Security breaches have potential to destroy company reputations, lose customers and revenue, and ultimately stop business. Just one of many examples includes Yahoo, who had 1bn email accounts compromised by biggest data breach in history and continue to battle questions around the security of customer data.
As application security becomes more and more of a critical concern for businesses, it’s going to become crucial that security is built, correctly, into the foundations of DevOps practices.
Whilst DevOps facilitates teams working together, the integration of DevOps and Security teams might require compromise from both sides to work together effectively.
Businesses that build in security at the beginning of the delivery process – at ground zero – and work to embrace the DevOps philosophies of teamwork, coordination, agility and shared responsibility, will be those that succeed in 2017.
How far away from Gartner’s DevSecOps are we?
At ECS Digital, we’re first to admit that most companies aren’t quite there yet. But, we’re getting there, and we believe that with a slight change in focus, DevSecOps will become a reality over the coming years.
Are you building security into your DevOps?
This blog was originally posted at ECS Digital.