When we built AppDynamics almost ten years ago, the APM market laughed at us. “No way, heard all these claims before,” “You can’t do that, it’s cheating,” “BS, no way you can monitor production in 4 minutes.” At that time, the monitoring market was littered with 30+ vendors and products, which, to be frank, weren’t up to the job. No wonder people didn’t believe in us.
If I told you that you could use Harness to build a fully automated canary deployment from scratch, in 4 minutes, would you believe me? Let’s give it a go and see what happens.
What is a Canary Deployment?
Canary deployments basically reduce the scope, impact, and risk of deploying new software artifacts to production. Instead of deploying a new artifact across all production nodes at the same time, you deploy to a subset of those nodes in phases, with each phase verified to ensure artifacts and your application are behaving appropriately.
For example, you might have a 3-phase canary deployment across your production stack like this:
After each deployment phase, you would verify the stability, performance, and quality of the artifact(s) and application before allowing the next deployment phase to execute. Most organizations today do this verification step manually with 4-6 engineers who each spend 30-60 minutes looking at monitoring tools and log files to make sure everything is OK.
In this blog, I’ll show you how you can automate this entire process in 5 minutes.
Step 1: Create New Application
Go to: Setup > Applications > Create New Application
It should take you 10 seconds to create a new application (a logical group to manage services and artifacts).
Step 2: Create New Service
Obviously, we need something to deploy, so let’s create a new service.
Go to: Setup > Your Application > Add Service
Enter My Microservice as the service name and choose Docker Image as the artifact type. Next, add the artifact source by referencing the specific artifact from your repository. This should take 20 seconds.
Step 3: Create New Environment
We now need somewhere to deploy our new service and artifact.
Go to: Setup > Your Application > Add Environment
Name your environment Production and choose production as the environment type.
Next, click Add Service Infrastructure, select My Microservice as your service, and then select Amazon ECS as the deployment type (Harness also supports Kubernetes). Next, choose your Cloud Provider account, region, and desired cluster to deploy to. This process should take 20 seconds.
Step 4: Create a Canary Workflow
Workflows tell Harness how to deploy services to environments.
Go to: Setup > Your Application > Add Workflow
Name your workflow. Select Canary Deployment as Workflow type and then select the environment you have just created (Production). This should take you 10 seconds.
Step 5: Create Pre-Deployment Steps
You’ll now be presented with a workflow wizard that will guide you through your canary deployment setup. Let’s add a simple pre-deployment step that will notify us via email that the canary deployment is about to start.
Go to: +Add Step under Pre-Deployment Steps and select Email
Enter details for fields To, Subject, and Body. This should take you 20 seconds.
Harness also gives you the option to invoke any 3rd party tools via HTTP REST in addition to native integration for things like Bamboo and Jenkins jobs, which you may want to kick off prior to any deployment.
Step 6: Create Canary Deployment Phases
Now comes the fun part. We get to play God and create canaries.
Let’s create a 3 phase canary deployment for our microservice that will deploy to 10%, 50%, and 100% of your environment respectively.
Click +Add Step under Deployment Phases to create Phase 1.
Pick the My Microservice and Production environment that you have just created. This should take 5 seconds. Now click Upgrade Containers and enter the number of desired nodes (10%) you want to upgrade in phase 1. This should take 10 seconds.
We now need to specify how Harness is going to verify the service that is being deployed as part of the canary phase.
Click +Add Verification and pick your monitoring tool(s), which is going to provide Harness with the data it needs to verify your service once it has been deployed.
Harness has out-of-the-box integrations with APM tools like AppDynamics and New Relic in addition to log tools like Splunk, Elastic, and Sumo Logic. This should take you 30 seconds.
For each verification source, you need to specify the target application details and default time period so that Harness’ unsupervised machine learning algorithms can immediately begin to process, analyze, and verify the data each of your verification sources provides.
Instead of humans manually looking at monitoring metrics or log data to verify each canary phase, Harness can automate this entire step completely by comparing the value of metrics and the contents of event data before and after each of your canary phases.
Repeat the above steps for canary phases 2 & 3 and your canary workflow wizard should look like this:
At this point, you have a few optional steps to consider:
- Notification strategy – what events in your canary workflow do you want to notify people with?
- Failure strategy – what steps do you want to take if any of your canary phase verifications fail? The default is an automated rollback.
- Workflow variables – do you want to “templatize” this canary workflow so other teams or services can use the same deployment workflow?
Once completed, your workflow should look something like this:
Step 7: Create a Pipeline for Canary Workflow
A pipeline in Harness is made up of one or more stages than execute in serial or parallel, with each stage referencing a specific deployment workflow. Most Harness customers have a 4 to 5 stage CD pipeline that allows them to deploy their services/artifacts across each of their dev, QA, staging, and production environments. For our canary deployment, let’s create a simple one-stage pipeline that invokes the Canary workflow we just created.
Go to: Setup > Your Application > Add Pipeline
Click the blue + round button, add a new pipeline stage called Production, and select the canary workflow we have just created. This should take you 30 seconds.
Step 8: Create a Trigger to Execute Our CD Pipeline
The final step is to now create a trigger that will execute the pipeline that contains our canary workflow. This is important because you may only want to trigger a production canary workflow under certain conditions.
Go to: Setup > Your Application > Add Trigger
Enter a name for your trigger, then pick the condition On New Artifact for your trigger, and finally, select the pipeline you have just created in Step 7. This should take you 1 minute.
That’s it! You’re all done with building your canary deployment.
How does Harness compare to your existing canary deployment process? If you’re interested in evaluating Harness, you can try our CD module today.
Cheers!
Steve
@BurtonSays
