Product
|
Cloud costs
|
released
January 4, 2021
|
3
min read
|

Vulnerability Scanning in your CI/CD Pipeline - Part Two

Updated

Implementing DevSecOps practices into your deployment pipeline is a prudent move. The adage that software components age like milk not like wine is pretty true. Sonatype, a vendor in the DevSecOps space has its popular Nexus IQ platform which is used for vulnerability scanning. 

Continuing on from part one of the series, we will be leveraging the Harness Platform to further operationalize vulnerability scanning in your CI/CD pipeline. The example is looking to deploy OWASP’s WebGoat Docker Image into a Kubernetes Cluster and interpret the scan results to deploy only if hygienic.

Like always, can follow along with the blog and/or video. All of the assets are in a GitHub repository.

Operationalizing Vulnerability Scans with Harness

For a first pass, up until this point shows that Nexus IQ can be called with no issue from Harness. Though if scaling this out to an organization, there will be a few more steps required. 

From the scan standpoint, will be masking the Nexus Credentials in the Harness Secrets Manager then prompting the user for a few details before submitting and interpreting the scan. 

To add a pair of secrets for the Nexus IQ user and password, leverage the Harness Secrets Manager.

Security -> Secrets Management -> Secrets -> Encrypted Text + Add Encrypted Text

Name: nexusiq_user

Value: admin

Once you click Submit, add another for the password.

Name: nexusiq_password

Value: admin123

With the secrets added, you can leverage Workflow Variables to prompt the user for certain parameters.

Setup -> Webgoat -> Workflows -> Scan Artifact. On the right hand side, click on “> Workflow Variables”

Click on the Pencil next to Workflow Variables to + Add.

For example, we can prompt the user for the Nexus IQ Application ID so that the scan gets related to the correct ruleset, etc. For this example we are only adding one variable, the art of the possible to parameterize all the inputs.

Variable Name: nexusiqappid

Type: Text

Default Value: webgoat-application

Once you click Save, you will see the Workflow Variable populated.

With Harness, you can reference Secrets with the syntax ${secrets.getVaule(“secret_name”)} and access Workflow Variables with ${workflow.variables.name}.

Update the Shell Script with the following updates.

Setup -> Webgoat -> Workflows -> Scan Artifact -> Shell Script

#Download and Scan Template
mkdir -p ~/nexus-scans
cd ~/nexus-scans
sudo docker pull webgoat/webgoat-8.0
sudo docker save webgoat/webgoat-8.0:latest > webgoat-8.0.tar
java -jar ~/nexus-cli/nexus-iq-cli.jar -i ${workflow.variables.nexusiqappid} -s http://<public_ip>:8070 -a ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -t build webgoat-8.0.tar

Click Submit and your changes have been saved. You can run through another Deploy to validate the changes worked. In the upper right of the Workflow, click Deploy.

Hit Submit and you will be running a scan in a more operationalized fashion.

Once we have the scan data, the next step is to interpret the results and then deploy. We can generate another pair of Harness Workflows (which we will later stitch together in a Harness Pipeline) to gather the results.

Create a new Workflow to interpret the scan results. 

Setup -> Webgoat -> Workflows + Add Workflow.

Name: Interpret Scan

Workflow Type: Rolling Deployment

Environment: Nexus IQ Server

Service: Nexus IQ Shell

Infrastructure Definition: Nexus IQ CentOS 

Once you hit Submit, create a Workflow to deploy Webgoat. You will come back and fill in the details for the scan interpretation soon.

Setup -> Webgoat -> Workflows + Add Workflow

Name: Deploy Webgoat

Workflow Type: Rolling Deployment

Environment: Webgoat Farm 

Service: Webgoat

Infrastructure Definition: Goat K8s Cluster

Once you hit Submit and return to the Workflow list, you will see three Harness Workflows.

The next piece is to build in some sort of interpretation logic to parse through the scan results. 

The next step will be to add a shell script to the Interpret Scan Workflow.

Programmatically Interpreting Scan Results 

Part of an automated DevSecOps practice is to influence the deployment if there are violating vulnerabilities found in a scan. At this time there is not a native integration between Nexus IQ and Harness, though can get access to and parse the scan results with a few cURL commands. 

The commands that are needed are getting the internal Nexus IQ App ID, the Report ID, and to count the violations in the report. If there are severe violations, stop the deployment. 

To start building this in, head back to Interpret Scan and add a Workflow Variable then a shell script in the Deploy Step similar to in part one.

Setup -> Webgoat -> Workflows -> Interpret Scan 

The first item to add is a Workflow Variable to prompt what would be a severe enough policy violation in Nexus IQ to stop the deployment.

Click on the pencil icon under Workflow Variables.

Variable Name: minfailpoilcyviolation

Type: Number

Allowed Values: 5,6,7,8,9

Default Value: 7

Required: True

Once you click Save, you can add the shell script under the Deploy Service in the Interpret Scan Workflow.

In the Add Step UI, search for shell and add Shell Script. Click next.

Assuming your user/password are still in the Harness Secrets Manager from part one, In the Configure Shell Script prompt, enter the following:

Name: Shell Script

Type: Bash

Timeout: 5m

Delegate Selector: NexusIQScan

Script:

#Interpret Scan Results - Template
echo "Interpreting Scan Results"

#Get AppID
export AppID=$(curl -u ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -X GET 'http:/<public_ip>:8070/api/v2/applications?publicId=webgoat-application' | jq '.applications[0] .id' | tr -d '"')
echo "AppID: " $AppID

#Get Report IDs
export ReportID=$(curl -u ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -X GET 'http://<public_ip>:8070/api/v2/reports/applications/'$AppID'' | jq '.[0] .reportDataUrl' | cut -d'/' -f 6)
echo "ReportID: " $ReportID

#Get Violation Count
export TotalViolations=$(curl -u ${secrets.getValue("nexusiq_user")}:${secrets.getValue("nexusiq_password")} -X GET "http://<public_ip>:8070/api/v2/applications/webgoat-application/reports/"$ReportID"/policy" | jq '.components[] .violations[] .policyThreatLevel' | grep  -c '[${workflow.variables.minfailpoilcyviolation}-9]')
echo "Total Sev Violations: " $TotalViolations

#Exit if Violations Match
if [[ $TotalViolations -gt 0 ]] ; then
 echo "Violations Preventing Deployment"
 exit 1
else
 echo "Moving Forward with Deployment"
 exit 0
fi

Hit Submit and your Shell and Workflow Variables should be present. Feel free to make any modifications to the script, was based on the Nexus IQ REST API.

The next step would be stitching all of the Workflows into a Harness Pipeline.

Automated CI/CD DevSecOps Pipeline - With Harness

So far our three separate Workflows are run independently of each other. By leveraging a Harness Pipeline, you can stitch together the three workflows together.

Setup -> Webgoat -> Pipelines + Add Pipeline. Name it “DevSecOps”.

Once you hit Submit, add Pipeline Stages to correspond to each Workflow.

Once you click + Add, the first step would be to scan the artifact.

Execute Workflow: Scan Artifact

Once you hit Submit, go through and add the interpret and deploy steps in a similar fashion.

Clicking through the two additional Stages. First, add Interpret.

Then, add Deploy.

Once you hit Submit with the Deploy Stage, your Pipeline will have three Stages.

You are now ready to run your Pipeline and see the fruits of your labor.

Running Your First Automated DevSecOps Pipeline

There are a few ways to run a Pipeline in Harness. Navigate to the left-hand navigation and select Continuous Deployment.

Continuous Deployment -> Deployments -> Start New Deployment

Pick a tag [latest] of Webgoat and hit Submit.

Once you hit Submit, you can watch your first automated DevSecOps Pipeline run. As expected, the Pipeline will fail because Webgoat has lots of violations.

Take a look at the Nexus IQ report also and see that the threats are there.

http://<public_ip>:8070 -> Reports -> Webgoat -> View Report

Harness, Your Partner in DevSecOps

Harness is here to partner with you no matter where you are in your DevSecOps journey. Application security testing methodologies are evolving with more items shifting left towards the developer and self-service. Having DevSecOps steps in your pipelines is a prudent move. Thanks for taking a look at the two-part series and make sure to sign up for a Sonatype Nexus IQ trial and a Harness account, today!

Cheers,

-Ravi

Sign up now

Sign up for our free plan, start building and deploying with Harness, take your software delivery to the next level.

Get a demo

Sign up for a free 14 day trial and take your software development to the next level

Documentation

Learn intelligent software delivery at your own pace. Step-by-step tutorials, videos, and reference docs to help you deliver customer happiness.

Case studies

Learn intelligent software delivery at your own pace. Step-by-step tutorials, videos, and reference docs to help you deliver customer happiness.

We want to hear from you

Enjoyed reading this blog post or have questions or feedback?
Share your thoughts by creating a new topic in the Harness community forum.

Sign up for our monthly newsletter

Subscribe to our newsletter to receive the latest Harness content in your inbox every month.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Continuous Delivery & GitOps