Integration of Security Practices in a DevOps Environment

Authors:

Ken Malmquist, Senior Director of Research and Development, Moss Adams

Altaz Valani, Research Director, Security Compass

Current challenges

Application security is a vital part of any organization’s business plan. While just about everyone will acknowledge this, a lot of times it isn’t given much further thought. Often, this deficiency will unfortunately only become painfully obvious when your security fails, and, by that time, the actual losses or reputational impact suffered by your business may be too great to recover from.

Finding the time and budget to do everything that is required to establish a baseline is hard enough. Keeping up with it in the midst of a fast-paced DevOps lifecycle– a problem often referred to as “DevSecOps”– only adds to the challenge. Either way, unless you’re starting from scratch, an adequate baseline assessment of your applications will be required as a starting point. Deciding what is “adequate” in this context will require some careful thought (the applications to assess, the security activities to execute, the tools to use, etc.) and will involve some tradeoffs which we will elaborate upon here.

Your first inclination may be to get a confirmation from the developers that their code is, in fact, secure. Several assumptions underlie this action: that developers understand how to code securely, that their test cases properly address security vulnerabilities, and they have the time to do this right. Beyond that, the temptation will be to shop around for the quickest and cheapest solution that seems to meet the minimum requirements. Unfortunately, this may or may not translate into actual security. It’s just like buying the cheapest possible health insurance. You may be perfectly happy with it until you actually get sick.

A simple, quick, dynamic scan or “smoke test” alone may provide some sense of comfort, but it is likely to be a false one. Any time-limited black box dynamic testing will leave considerable uncertainty on the table because a determined adversary has nearly unlimited time to attack your application. White box evaluations with access to source code are much better, but automated scanning of the code is usually not adequate to capture all potential vulnerabilities. Even the best automated scanner isn’t going to catch everything, especially flaws that involve errors in logic. In most cases, only the combination of automated static and dynamic scans with some amount of manual code review can provide you with a fully accurate indication of the state of your application’s security.

A good threat model and a security-focused business logic review are highly desirable. Ideally, you can do this up-front, before coding actually begins. But whether starting from scratch or departing from some known baseline, these artifacts will greatly help reduce code scanner noise going forward by fine-tuning your scanners to only identify issues that are important to your specific security context.

Once the baseline is in hand, it is important to introduce key security activities to the DevOps pipeline itself. These must be included in a natural way that doesn’t slow the developers down because DevOps is, after all, about speed. Scheduled static and dynamic analysis scans, for example, should be integrated with your continuous integration (CI) server, such as Jenkins, to run automatically with each new build. These scans themselves should be configured with custom rules and filters to report on only the issues that you have decided are of concern to you and to reduce false positives. Software Composition Analysis (SCA) scans may be integrated in a similar manner to provide timely feedback on what Open Source Software (OSS) is being introduced and the security and licensing risks that this might impose.

Another modern security technology that you should consider introducing into your DevOps pipeline is Interactive Application Security Testing or IAST. IAST works via bytecode instrumentation. Although that may sound involved, it really isn’t. For example, in a Java application, it can be as simple as deploying an additional JAR. Once deployed, IAST monitors your application through the course of the normal exercise of its functionality, for example during QA testing. Because of the instrumentation, it can detect and give immediate feedback on relevant security issues not only in your custom code but also in included third-party and open source library code as well.

DevOps is all about thinking holistically, and DevSecOps needs to be as well. This means that security engineers are part of the team and security is addressed in a proactive fashion, as opposed to dealing with what’s wrong when you think you’re finished. You know you’re making progress when your team starts thinking of security as a feature to be implemented, as opposed to a bug to be fixed.

What we need


Focus on developer training.

The developers themselves are the first defense. The least vulnerable security flaw is the one that’s never introduced in the first place. The problem is developers do not know this stuff already. They do not always learn it, or learn it adequately, in school. You need to invest in developer training which will pay dividends downstream and you’ll be able to gain confidence that they really do know what they’re doing. A lot of the required information is freely available online, for example from the Open Web Application Security Project (OWASP) and from NIST. Instructor-led or computer-based training can supplement this information and make it more easily digestible.

Provide quick and relevant feedback.

Of course, even a knowledgeable developer is going to make mistakes, and it is crucial that in a DevOps environment they get nearly instantaneous feedback when that happens. Regular scanning by static analysis tools is one such feedback loop, so long as the feedback from this process goes directly into the environment and workflow that they already live in. In the ideal case, this means that an IDE plugin will show them the problem directly in their editor and that items for remediation will go directly to their defect tracking system. The worst possible case would be to hand them a multiple page PDF document with a list of issues!

Automate.

Oftentimes, lightweight “lint checkers” will provide more timely and immediate feedback than an automated and scheduled static analysis scan, and there’s no reason not to use both. The goal here should be to automate your pipeline in the development and operations domains as much as possible.

Build for traceability.

Security activities across the DevOps lifecycle need to be traceable. Having security requirements being forward and backward traceable to security test cases (both functional and non-functional) can demonstrate how well the requirements have been met.

Conclusion

Start with a baseline. Perform a baseline application security assessment to allow immediate emergency remediation action if any critical vulnerabilities are uncovered.
Prioritize your security activities. Not all security vulnerabilities that an auditor, penetration tester, or an automated tool will call out are of equal priority. Many times, policy decisions regarding acceptable risk from a business standpoint must be carefully considered and established upfront, so that time and money aren’t wasted tracking down things that really aren’t that important.


Work longer term on building a culture of security. Work toward moving from a reactive posture to a more proactive one that meets the needs of the business. Keep in mind, the real world is not ideal and integrating this new security culture and mindset into an existing process can’t be done all at once. Grow this step by step.

Previous Article
Security does not inhibit DevOps
Security does not inhibit DevOps

Learn why security does not get in the way of DevOps.

Next Article
Mission impossible? The benefits of standardizing on DevSecOps
Mission impossible? The benefits of standardizing on DevSecOps

Learn about the benefits of standardizing on DevSecOps in your organization.