Harness CI With AWS VMs

Harness CI recently added beta support for AWS virtual machines (VMs). This addition means that CI builds can run on either a Kubernetes cluster or AWS VM in the Harness platform. In this post, we’ll explore the features supported in AWS VMs and explore their design and architecture.

The AWS VM feature comes with Test Intelligence support for .NET. It supports all of the features that are already present in Harness CI using Kubernetes infrastructure. A major addition is the support for running builds on Windows OS. Apart from this, the run step can also execute commands on the host VM. Therefore, AWS VM is a default choice for CI pipelines that are unsuitable for execution inside containers.

Moreover, steps can run Docker commands on pipelines using AWS VM. This may not be feasible on a Kubernetes infrastructure with restrictions on privileged access to pods.

Design

Every CI stage runs on a dedicated AWS VM that is terminated after the build execution is complete. A major advantage of this approach over reusing VMs for multiple builds is that workspace cleanup isn’t required, since every build runs on a new VM.

This feature requires the installation of Drone Runner AWS on a delegate VM. The steps to install a delegate with a runner can be viewed here. The runner is responsible for the lifecycle management of AWS VMs that include creation, deletion, etc. Furthermore, it handles the execution of steps on the VM where the CI build runs. The runner starts an HTTPs server at startup, and the delegate agent interacts with it over localhost on port 3000.

A CI build execution comprises three phases: initialization, step execution, and cleanup. All of these phases happen via delegate tasks.

Initialization

This phase creates the resources required for a build to run. Once the initialization task reaches the delegate agent, it calls runner to create a VM on which the build will run. 

Runner creates an AWS VM using the AWS golang sdk. As part of VM creation, it also installs an agent, in particular lite-engine on the build VM. The lite-engine installation happens via a cloud init script that downloads it from the cloud and starts it on VM startup. Lite-engine starts an HTTPs server, which is used by the runner to communicate with it over a secured channel. 

The lite-engine HTTPs server uses self-signed certificates with both server and client certificates. Client certificates reside on the runner and are used to authenticate the client (runner here) identity to the server. And server certificates present on the lite-engine are used to encrypt data-in-transit.

Step execution

In the step execution phase, the steps are executed in the order that is defined in the CI pipeline. Once a task to execute a step reaches the delegate, it calls the runner to execute it. Then, runner calls lite-engine over HTTPs to execute the step. Lite-engine either runs the step directly on the VM as a sub-process, or it executes it on a docker container.

Cleanup

The cleanup phase marks the completion of a CI stage. It deletes the resources allocated for the build execution. The delegate agent, upon receiving the cleanup task, calls the runner, which deletes the AWS VM that was allocated to the completed build.

VM Pool

VM bootup can take a couple of minutes especially for windows VM which takes 6-8 minutes for startup. If the VM is created as part of the build execution, then it can significantly increase the build time due to the VM bootup time.

Harness uses a pooling strategy that maintains a user-specified number of warm AWS instances. Warm instances are pre-initialized Amazon Elastic Compute Cloud (Amazon EC2) instances for which VM startup is already complete. Whenever a pipeline with the CI stage is executed, the runner on initialization task checks whether an AWS instance is present in the pool. If it’s present, then VM is detached from the pool and assigned for the CI stage execution. Runner creates a new Amazon EC2 instance to maintain the size of the pool. Therefore, the VM pool makes sure that the initialization takes seconds to complete rather than minutes.

The VM pool must be defined as part of the runner configuration file. It allows for setting the size of the pool, as well as the configuration of the VM, including AMI, instance type, network, etc.

Conclusion

Harness CI with AWS VM provides the flexibility to run Linux and Windows builds on Amazon EC2 instances. We hope that this high-level design overview provides insights into how Harness CI works with AWS VMs.

Want to see it for yourself? Sign up for your free trial of Harness CI now!