Top 10 Best Practices for DevSecOps

Software development teams are challenged to maintain high velocity while also ensuring that they minimize the number of software vulnerabilities in production services.

Why Is DevSecOps Important and What Exactly is DevSecOps? 

Traditional application security approaches have struggled to keep up with the rapid pace of software delivery, so companies have begun to use security practices that implement DevOps engineerings concepts. Using this approach, development teams can achieve high velocity software delivery with developer-first security and governance baked in.

The DevSecOps framework can lead to great results, but as with most IT disciplines, there are pitfalls to avoid. Understanding and implementing DevSecOps best practices is the key to avoiding those pitfalls.

‍
So What is DevSecOps Engineering?

It is a software development approach that incorporates security practices into the entire software development lifecycle, from design to deployment and maintenance.

The term "DevSecOps" combines "Development," "Security," and "Operations" to emphasize the importance of security in the development and operation of software systems. DevSecOps engineering aims to integrate security into the development process, rather than treating it as an afterthought.

By implementing security measures early in the development process, DevSecOps engineering helps to identify and address security vulnerabilities before they can be exploited. This approach also promotes collaboration between development, security, and operations teams to ensure that security considerations are taken into account at every stage of the development process.

Overall, DevSecOps engineering is an approach that prioritizes security in software development, ensuring that security is built into software from the beginning and maintained throughout its lifecycle.

‍

Intro to DevOps Security

What is DevSecOps methodology? 

DevSecOps methodology is the philosophy and practice of adding security automation across the software delivery life cycle (SDLC) in harmony with all phases. Most IT professionals are familiar with the DevOps framework infinite loop diagram. We can map the phases of the loop to their practical implementations:

• Build → Continuous Integration (CI Pipelines)
• Test → Part of Continuous Integration, Continuous Delivery, and Continuous Deployment
• Release → Continuous Delivery (CI/CD)
• Deploy → Continuous Deployment (CD pipelines and Feature Flag Management)
• Operate → Continuous Deployment and Production Operations

You might think that DevOps security (aka shift-left security) is only part of the Test phase, but DevSecOps best practices are a shared responsibility and should be spread across all phases to ensure the best results.

At its core, DevSecOps is a methodology where engineering teams run security scans throughout the software delivery life cycle to find and fix vulnerabilities before they make it to the end user. Let's go through DevOps security best practices.

How Does DevSecOps and Security Testing Orchestration Relate?

Security Testing Orchestration, on the other hand, is the process of automating security testing activities and integrating them into the development process. It involves the use of tools and technologies to streamline and standardize security testing, making it easier for development teams to identify and address security vulnerabilities.

In practice, DevSecOps and Security Testing Orchestration work together to improve the security of software development. DevSecOps ensures that security is a priority throughout the development lifecycle, while Security Testing Orchestration automates and streamlines the security testing process, making it more efficient and effective.

By integrating Security Testing Orchestration into the DevSecOps approach, development teams can identify and address security vulnerabilities earlier in the development process, before they become more difficult and expensive to fix. This approach helps to ensure that software is secure by design and reduces the risk of security breaches and other security-related issues.

‍

Best Practices for DevSecOps Security

Increase Collaboration Between Developers and Security 

Just as the DevOps philosophy promotes breaking down the barriers between engineering and operations teams, the DevSecOps principles promote breaking down the barriers between the engineering and security teams.

Promote a Culture of Security Testing 

As with DevOps, DevSecOps requires a cultural shift to be successful. There are two ways that changes to corporate culture can be made:

1. Mandated change from top down – typically the executive staff will communicate the required changes across the organization.
2. Organic change from bottom up – cross-team security collaboration starts small and expands to other teams as word of success spreads.

Neither of these approaches is easy to implement, but both can be effective at creating a culture focused on identifying and resolving security issues before they reach the end user. Each organization needs to determine what approach works best for them, and many organizations seek to implement a synthesis of both tactics.

Automate All Security Scanning

DevOps teams have adopted an approach of automating everything they can related to CI/CD pipelines. The same principle applies to creating secure applications using DevSecOps practices. Carefully examining all steps in the secure application delivery workflow helps determine what can be automated. Then you can create a prioritized list based on importance and required effort. Work down the list until everything that makes sense has been automated.

Implement Automated Security Scanner Governance

Governance is an important topic across all enterprises these days. Compliance initiatives mandated by regulatory agencies require enterprises to track and document their security measures. Using the right tools, organizations can implement policy as code (Open Policy Agent is one example) to enforce and document the usage of approved software security scanners. This has the benefit of standardizing the use of scanning tools while also making it faster and easier to pass compliance audits.

Add Security Guardrails to the Development Process

It's not enough to require and enforce the use of standardized software security scanning tools. It's also important to provide security gates or guardrails that are informed by the output of the scanners. Here again, policy as code can be used to control the workflow of CI/CD pipelines using the real-time scanning tool output.

Treat DevSecOps as a Big Data Problem

The DevOps movement led to a proliferation of new tooling, which has created significant amounts of data. In a similar fashion, the DevSecOps movement is necessitating more tooling with massive amounts of data output. The analysis of the tooling output is critical to minimizing business risk by reducing production application security vulnerabilities. Big data techniques like normalization, deduplication, compression, correlation, etc. should be used to quickly and efficiently analyze the combined set of scanner results.

Reduce the Workload on the Engineering Team 

The definition of DevSecOps could be as simple as "shift-left application security." While this is oversimplified, it does highlight the fact that DevSecOps shifts more workload to the engineering team –more specifically to developers. With this in mind, it's imperative that adoption of DevSecOps practices do not overload the engineering team. 

We've already discussed the "big data" result of adding multiple security scanners across CI/CD pipelines. Without providing adequate tooling, the developers will have to manually perform the data analysis to determine what vulnerabilities should be fixed, the priority of fixes, and perform all of the associated workflows (creating Jira tickets, tracking exceptions, etc). Without reducing this burden on the engineering team, DevSecOps will be significantly more difficult to implement. Naturally, the next question is "How do we reduce this burden on our engineering team?" The answer is, by using the right combination of tooling and workflow. Check out our DevSecOps Checklist for guidance on choosing the right tooling. 

Automate Ticket Creation and Tracking 

Every detected vulnerability should have an associated Jira ticket. For the efficiency of your engineering and IT security teams, these tickets should be created automatically. Even in the case where detected vulnerabilities are false-positive, you still need a ticket for audit purposes. The right tooling will create these Jira tickets for you and associate them with the application services where the vulnerability was detected. When the vulnerability is remediated (and the fix is verified), the ticket should get updated and closed automatically.

Update Your Tools as Needed 

Almost every best practice discussed so far requires tooling to assist with automation or analysis. Traditional IT security tools were not designed with modern DevOps practices and cloud-native technologies in mind. Companies should re-evaluate the required tooling to ensure their DevSecOps program is successful. 

Many organizations wrestle with the build vs buy decision. Building complicated tooling for an ever-changing landscape is typically not practical to maintain for most companies. Therefore, as you evaluate tools that fit your organization’s needs, you may find our DevSecOps checklist to be a useful reference. 

Invest in Workflows 

What happens when a new vulnerability is detected? What is the process for requesting and tracking a security exemption? How do you inform other teams of security exemptions? Some of these processes and workflows may already exist in your organization, but some may need to be created. Whether already existing or creating new, these workflows all need to be automated to scale the security practice across your organization.

Consider the Full Life Cycle of Your Apps 

If you're already adopting DevSecOps, you understand that only running security scans after an artifact gets to production is not good enough. Security scanning must be implemented across the full life cycle of your application services. 

• CI Pipeline
• Software Composition Analysis (SCA) - identify vulnerabilities in open-source software or third-party components used to build application services.
  Examples of SCA tools: Black Duck SCA, Snyk, WhiteSource SCA.
• Static Application Security Testing (SAST) - perform static code analysis on an application's source code or bytecode.
  Examples of SAST tools: Brakeman, SonarQube, ShiftLeft.
• CD Pipeline and Production Operations
• Infrastructure as Code (IaC) Testing - a subset of infrastructure security, vulnerability assessment of software-defined compute, network, and storage.
  Examples of IaC testing tools: Checkov, TFLint, Terrafirma.
• Dynamic Application Security Testing (DAST) - analyze the application at run-time to determine if there are vulnerabilities.
  Examples of DAST tools: MicroFocus Fortify, OWASP ZAP, HCL AppScan.
• Interactive Application Security Testing (IAST) - combine elements of SAST and DAST, typically within a JVM or CLR.
  Examples of IAST tools: Checkmarx IAST, HCL AppScan Enterprise, Synopsys Seeker.
• Container Security Testing - security analysis of application containers and Kubernetes configuration.
  Examples of container testing tools: Qualys, Docker Bench for Security, Aqua Container Security.
• Fuzz Testing - intentionally feeding bad information into running applications to determine security vulnerability status.
  Examples of Fuzz testing tools: Beyond Security beSTORM, Synopsys Fuzzing Test Suite, Google OSS-Fuzz.
• API Testing - specialized testing of API endpoints to identify security problems.
  Examples of API testing tools: Apigee, Katalon Studio, Astra.

Conclusion: Keep Up With the Rate of Change by Updating Security Tools and Processes

DevSecOps is far more than just getting developers to run security tests. It's about automating security with a developer-first mindset, expanding information security beyond a small team of security experts, and reducing the security risks that make it to the end user. The big challenge is to accomplish all of this without reducing the speed of software delivery.

With the proliferation of cloud-native technologies and open-source components used within applications today, security tools and processes must be updated to keep up with the rate of change. Traditional security tools and processes are not fit for purpose in modern environments.

Harness Security Testing Orchestration (STO) can help you implement the best practices discussed in this article. STO seamlessly integrates security scanning across the software delivery life cycle, provides policy-as-code governance, and performs the big data analysis required to make your DevSecOps program successful. Are you ready to adopt DevSecOps practices? Download our checklist outlining the key capabilities you’ll want to have covered.