DevOps & Software Security: Turning unplanned work into planned work

Every IT worker I’ve met has heard me rave about The Phoenix Project. The book uses an all-too-realistic fictional scenario to discuss the behaviors of a high performing IT organization, with a particular emphasis on the convergence of development and operations (i.e. DevOps). One lesson from the book that really resonated with me was breaking down IT work into four types:

  • Business projects
  • IT projects
  • Changes
  • Unplanned work

For as long as I can remember, most of software security work has fallen under the scope of the last bullet: unplanned work. The book illustrates how unplanned work is one of the most destructive forces in IT. A security defect detected through verification activities such as static analysis or penetration testing is unplanned work by definition: developers need to stop working on some planned piece of functionality in order to remediate the issue. Simply building and then fixing recurring security defects does not hold true to the spirit of continuous process improvement illustrated by high performance IT organizations. It means higher costs for development and often creates tension between security and development teams.

To me, unplanned work is the most important reason to have a repeatable, scalable process for security and other non-functional requirements. Specifying the right requirements up-front — either in a waterfall requirements phase or as part of agile iteration planning — allows project teams to explicitly agree on which security risks they will prevent and which ones they will accept into production. Explicitly identifying security requirements allows developers to hone in on potential false negatives from security scanning solutions. They can then use scanning tool customization or automated front-end / unit-testing to detect potential vulnerabilities not covered by their scanning solution. In addition, they can build controls in development frameworks to eliminate the risk entirely such as context aware output encoding for cross site scripting.

IT teams that practice high performance DevOps activities lean heavily on automation technologies like static analysis, configuration management, and continuous integration. Adding non-functional requirements to the mix will help round out coverage gaps from other automation techniques, and turn unplanned work into planned work.