Migrating Clouds With Harness Continuous Delivery

About a year ago Harness migrated public cloud providers, and to our teams surprise, the process took just 76 minutes, all without a single hiccup or sneeze.

Here's what our #engineering Slack channel looked like:

At 8:12pm on June 5th 2018 our VP Customer Success Ohad (he loves ice cream) asked our engineering for a status update, a few minutes later our engineer Brett kicked off the migration via a Harness deployment pipeline (Harness uses Harness for Continuous Delivery). 76 minutes later the deployment pipeline completed and the DNS was switched. No explosions, electric shocks, thunder or lightning. It was the most uneventful cloud migration you could have witnessed.

So, how did we do this?

Simple. We ate our own dog food.

In a nutshell: Harness allows you to automate the deployment of software artifacts to any cloud or data center infrastructure, and it does this using abstraction.

In this post, I'll introduce the Harness CD Abstraction Model (CDAM) and then summarize the different cloud migration strategies which the model can help you with.

The Harness CD Abstraction Model (CDAM)

One of the key architecture decisions behind Harness was to provide customers with abstractions between their services, infrastructure, and toolchains.

Dependencies are evil; the more your deployment pipelines are service/infra/tool specific, the longer it takes to migrate them should you wish to re-platform or retool.

An example of this is Jenkins pipelines, because all underlying scripts/jobs tend to be service/infra/tool specific, meaning deployment pipelines are hard-wired (e.g. AWS AMI deployments only). They become "brittle" and "difficult to scale" as services evolve over time and incorporate new technology and tooling.

To minimize dependencies and improve velocity, we created the Harness CD Abstraction Model (CDAM):

In short, our Continuous Delivery platform is not service, infrastructure, or tool dependent.

Here are a few examples:

Services - Harness supports a wide variety of artifacts (Jars, AMIs, containers, lambda functions, ...) and repositories (Artifactory, Nexus, Jenkins, S3, DockerHub, ...).
Provisioners - Harness supports AWS CloudFormation, HashiCorp Terraform and custom scripts for provisioning infrastructure.
Cloud Providers - Harness supports every major cloud provider and supports traditional on-premises clouds/DCs.
Verifications - Harness supports all major monitoring, APM and Log management tools, in addition to having a custom API.
Secrets - Harness supports HashiCorp Vault, AWS Secrets Manager and also provides native Secrets Mgmt via AWS KMS.
Deployment Strategies - Harness supports rolling, multi-service, blue/green and canary deployment strategies.
Notification Strategies - Harness supports Email, Slack, HipChat, PagerDuty, Jira, ServiceNow.

The bottom line is that you can switch in and switch out services, infrastructure, tools, and strategies without having to rewrite all your deployment pipelines. Harness uses abstraction and APIs to do the integration work for you so you don't need to script it yourself.

Since Harness exited stealth in October 2017 companies like Citi, SoulCycle, and Advanced Software have all dramatically accelerated their cloud migrations using the Harness CDAM.

Lifting and Shifting

Lift and shift is perhaps the easiest migration strategy to understand and execute.

Stick your apps/services in a wrapper VM, AMI or container and throw it on any cloud compute. Job done. This works well with standalone services that run on platforms like Docker, Kubernetes, Microsoft IIS, Apache Tomcat or JBoss. The only thing that changes is the underlying compute resource.

To do this in Harness:
Define the Service (artifact type) and its location (repository):

Define the target Environment (service infrastructure/cluster/region/load balancer & cloud provider):

Define the Workflow (deployment & release strategy):

Now run the Harness workflow to lift and shift your service. It's that simple.

Re-Platform

Our Harness cloud migration was actually a re-platform migration strategy. Harness by design was built as a Software as-a-Service (SaaS) platform but as we engaged with several enterprise buyers, the demand for an on-premises offering increased. We, therefore, used our cloud migration as a means to re-platform to a more cloud-native containerized Kubernetes application so we could benefit from enhanced portability, management, and scaling. It would also allow those few enterprise customers who requested Harness on-premises to run on whatever Kubernetes cluster or infrastructure they wanted.

Re-platform is similar to Lift and Shift; the only difference is that you're probably changing the artifact type of your service. In our case, Harness went from deploying a few native jars in our deployment workflow to a single container. We also changed our cloud provider from X to Y as well as the environment cluster.

The good news is that Harness provides first-class citizen support for Kubernetes, and also queries your cloud provider account so clusters, configuration, and meta-data for environments are dynamically populated. It's a 30-second task to switch cloud providers and clusters. You don't have to write any scripts, code or configuration to re-platform and redeploy your service, Harness provides all that.

Onboarding New Cloud Services

Onboarding new services (e.g. Kubernetes microservices) is probably the most common migration strategy we see at customers.

Typically when customers are making the transition from a monolith to a microservices architecture they need to build new deployment pipelines to "onboard" these services. Instead of spending weeks to script and onboard using Jenkins Pipelines, customers can achieve this in just a few hours with Harness.

For example, OpenBank (The digital bank of Santander) said "Our Jenkins Pipeline jobs were too painful to manage and deploy with."

Here's how to build a Kubernetes pipeline in 5 minutes:

Hybrid Cloud Architectures

With Harness, it's super easy to define a single environment constructed on multiple services (and hybrid clouds).

For example, let's suppose we create a new production environment (below) which has 3 microservices, and each microservice runs in a different cloud (AWS, Azure, GCP). We can define that environment in less than 2 minutes using the Harness environment wizard:

Hybrid clouds in Harness aren't just limited to public cloud providers either, you can specify private clouds or bare metal infrastructure for on-premises data centers.

Obviously, Harness isn't going to be able to refactor or rearchitect your existing monoliths or legacy apps to make them cloud-native. Our engineers haven't got around to that Jira ticket titled 'auto-refactor magic' yet. We can but hope.

In the meantime, happy cloud hopping.

You can sign up to try Harness.
‍