Deployment Strategies
For Continuous Delivery

Know what a basic blue/green rolling canary deployment is? Unfortunately, that one doesn't exist, but here are a few that do.

By Steve Burton
February 2, 2018

You might hear terms like “blue/green” and “canary” when it comes to the final act of deploying code in production.

Pushing the deployment button these days is a bit like playing craps; you roll the dice and it’s all about staying alive. There are, however, a few deployment strategies you can do though to increase the odds of staying alive. Below are a few examples of what to do and what NOT to do.

Basic Deployment (aka Suicide)

All nodes within a single environment are updated at the same time with a single new service/artifact version.

When to use it:

  • Your app/service is not business, mission or revenue critical OR
  • You’re deploying out-of-hours and no one is using the app/service OR
  • Your ass won’t get fired if the app/service fails OR
  • You’re still living in the stone-age

Pros: Simple, fast, cheap

Cons: Risk, outages, slow rollback, unemployment

I laugh at this now, but when I was a developer back in 2004 this is how we rolled. We’d basically get someone in Ops to update the servers at midnight and then all pray before we went to bed. Hopefully, when we showed up at work the next morning, everything was good and no one was in trouble.

Multi-Service Deployment

All nodes within a single environment are updated at the same time with multiple new services/artifacts.

When to use it:

  • When your application has service/version dependencies
  • You’re deploying out-of-hours and no one is using the app/service

Pros: Simple, fast, cheap, less risk than basic deployment.

Cons: Risk, difficult to test/verify all service dependencies, outages, slow rollback

Rolling Deployment

All nodes within a single environment are incrementally updated one-by-one or in N batches (as defined by a window size) with a new service/artifact version.

Pros: Simple, cheap, relatively simple to rollback, less risk than basic deployment.

Cons: App/DB needs to support both new and old artifacts. Manual checks/verification at each increment could take a long time.

Blue/Green Deployment

Two identical environments called “blue” (aka staging) and “green” (aka production) run simultaneously with different versions or service/artifact. QA and UAT are typically done on the blue environment. When satisfied, traffic is flipped (via a load balancer) from the green environment (current version) to the blue environment (new version). You can then decommission the old environment once deployment was successful.

Note: Netflix calls this a red/black deployment. Presumably this nomenclature is enforced by their corporate marketing team? 🙂

Pros:

  • Simple, fast, well understood, and easy to implement
  • Less risk relative to other deployment strategies
  • Rapid rollback (flip traffic back to old environment)

Cons: 

  • Replicating a production environment can be complex and expensive (i.e. microservice downstream dependencies)
  • QA/UAT test coverage may not identify all anomalies & regressions in blue environment
  • An outage or SPOF could have wide-scale business impact before rollback kicks in
  • Maybe lose certain in-flight user transactions when the flip is made
  • Database compatibility (schema changes, backward compatibility)

Canary Deployment

All nodes in a single environment are incrementally updated in small phases, with each phase requiring a verification/gate to proceed to the next phase.

This is currently the most fashionable way to deploy apps/services into production.

Pros:

  • Deploy in small phases (e.g. 2%, 10%, 25%, 50,%, 75%, 100%)
  • Lowest risk relative to all other deployment strategies (reduce business exposure)
  • Test in production with real users & use cases
  • Run & compare two service versions side-by-side
  • Cheaper than blue/green, no need to have two production environments
  • Fast and safe rollback

Cons:

A/B Testing:

Different versions of the same service/artifact run simultaneously as “experiments” in the same environment (typically production) for a period of time. Experiments are either controlled by the deployment of distinct artifacts or through the use of feature flags/toggling and/or AB testing tools (e.g. Optimizely).

User traffic is commonly routed to each different version/experiment based on specific rules or user demographics (e.g. location, interests, …). Measurements and comparisons are then performed across experiments to see which returned the best result. After experiments are concluded, the environment is typically updated with the optimal service version/experiment.

The biggest difference between AB testing and the other strategies is that AB testing deploys many versions of the same service/artifact to an environment with no immediate goal of updating all nodes with a specific version. It’s about testing multiple ideas vs. deploying one specific tested idea.

Pros: Fast, easy and cheap way to test new features in production. Lots of tools exist to enable this.

Cons:

  • Experiments can sometimes break app/service/user experience.
  • Scripting AB tests can be complex.
  • Database compatibility (schema changes, backward compatibility)

Which Deployment Strategy Should I Use?

It depends entirely on the type of application/service and environment. Based on my conversations with customers, the majority of teams are currently using blue/green or canary deployments for mission-critical web applications. In many cases, customers are migrating from blue/green to canary so they can test in production with minimal business impact.

You can also combine many of the above deployment strategies into a single strategy. For example, at Harness, we have customers doing multi-service canary deployments.

If you want to test or implement any of the above, feel free to request a trial of Harness.

What deployment strategies are YOU using? and why?

Cheers,

Steve.

@BurtonSays

 

➞ Back to Blog

Leave a Reply

avatar
  Subscribe  
Notify of