CloudFormation and Terraform for Harness - Adding Infrastructure Provisioners

Authors:

Table of Contents

Infrastructure provisioners are a big deal for cloud-native applications and environments. Automating the provisioning and decommissioning of infrastructure allows you to scale up and down on-the-fly while reaping the cost benefits of on-demand computing.

By integrating this process into your deployment pipelines across all environments, you can effectively scale end-to-end automation and gain cost benefits for all dev and engineering teams.

For example, only spin up a dev, QA, and staging environment when a new build or version exists or needs testing. You can also use them for rapid blue-green deployments when you need to replicate production.

Over the past year, we've consistently heard from customers that AWS CloudFormation and HashiCorp Terraform are the new standards; as a result, integration with the Harness Software Delivery Platform is a "no brainer."

What Is AWS CloudFormation?

AWS CloudFormation provides a common language for you to describe and provision all the infrastructure resources in your cloud environment.

CloudFormation allows you to use a simple text file to model and provision, in an automated and secure manner, all the resources needed for your applications across all regions and accounts. This file serves as a single source of truth for your cloud environment.

What Is HashiCorp Terraform?

HashiCorp Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open-source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.

Terraform is like AWS CloudFormation, but is multi-cloud - meaning it's a common language for defining infrastructure across AWS, Azure, GCP, and so on.

Introducing Harness "Infra Provisioners"

To accommodate CloudFormation and Terraform scripts/templates, we introduced a new concept called "Infrastructure Provisioner" into our deployment pipeline model:

Infrastructure provisioners can be added to any deployment Workflow so that infrastructure (cloud clusters) can be provisioned and decommissioned (if needed) as part of the deployment process.

The idea is that you can easily reuse, manage, debug, and orchestrate CloudFormation and Terraform scripts/templates across your deployment workflows and pipelines. Your infrastructure stays in sync with what's happening at the application level.

Creating an Infrastructure Provisioner

Go to: Setup > Your App > Infrastructure Provisioners
Click + add Infrastructure Provisioner and select either Terraform or CloudFormation.

Next, add the body of your script/template, or simply reference it via the source type dropdown:

You can automatically parse input variables from scripts/templates by pushing the populate from template control, or you can manually create them using the +Add control.

Next, you need to map each infrastructure provisioner to one or more services (artifacts). Click Add Service Mapping and select the required service, deployment type, and cloud account. You can then specify appropriate cluster config, either as literals or variables using the $ prefix; this basically allows you to pass parameters from CF/Terraform into Harness workflows & environments.

Your infrastructure provisioner setup is now complete:

Adding Infrastructure Provisioners to Deployment Workflows

Once your infrastructure provisioners are defined, you can now add and reference them in any deployment Workflow.

For example, below is a simple canary deployment workflow with 3 phases.

We can easily add/reference our CloudFormation or Terraform scripts/templates by adding a Pre-Deployment Step to our workflow:

This control presents us with the following dialogue where we can add our CF/TF infra provisioner to the workflow:

Now you can instantiate your CF/TF script/template using the relevant cloud provider details in addition to script/template variables, that can either be static with literals or dynamic with parameters.

Once added, your workflow will look like this showing that your infra provisioner will be executed prior to deployment phase:

You can also add CF/TF as a Post-Deployment Step if you want to use them to decommission infrastructure as well.

Debugging CloudFormation & Terraform Deployments

Now, when you execute a Pipeline or deployment workflow, you'll see all the CloudFormation and Terraform console output in the context of the deployment. This is useful for debugging and managing any infrastructure provisioning issues: