Simplifying AWS Fargate deployments with Harness
The container conundrum is not an easy one to solve. We just went on a six part journey on a specific container orchestration tool, Kubernetes (K8s). In those six parts we went through the lineage, opinion, and a handful of use cases for K8s; but there was a piece missing. We did not give much mention about the actual infrastructure that an orchestrator and the images/containers reside on. Building the infrastructure that supports cluster infrastructure itself is a complicated undertaking; for example, the AWS infrastructure to run your own Kubernetes Cluster.AWS Fargate hopes to remove complexity in managing clustering technology and the infrastructure running the cluster(s) with a prescription from AWS. Fargate provides a service orchestrating and packaging several underlying AWS services to remove complexity.
Is Kubernetes too complex?
Building the container image is just one part of the equation. After spending some time Dockerizing your application, time to unleash your hard work to the world. At a bare minimum you need a spot to execute Docker Run or your container run-time of choice. Leveraging Kubernetes is a design and architectural decision.Running your own Kubernetes Cluster on top of a public cloud vendor can prove to be a learning curve not only with Kubernetes itself but making sure the infrastructure is prudent to a Kubernetes Cluster. Wading through the instance types, scaling groups, storage, and networking rules can take valuable cycles away from innovation.Most likely there will be a need for a few different Kubernetes environments. From local on your desktop in the flavor of Minukube to lower environments to build and test your applications on before releasing them into the wild.There is an expectation that there will be a resource maintaining the Kubernetes cluster(s) and expertise in Kubernetes. AWS has Elastic Kubernetes Service (EKS) which is AWS’s managed Kubernetes service. Maintaining and administering a Kubernetes cluster especially at scale can be difficult. Proof in point, AWS themselves with EKS still runs older versions of Kubernetes than the latest; at the time of this blog, Kubernetes 1.15 is out and the latest supported in EKS is 1.13.7.To take the complexities of Kubernetes out of the picture, Fargate though is not Kubernetes based; which could come as a surprise if looking to build something today.
Fargate is not Kubernetes based - Hello Tasks
When I first heard of Fargate at AWS re:Invent in 2017, I immediately assumed that Kubernetes was somewhere in the mix and kept looking for the K8s logo in Fargate material. This being the third container service after Elastic Container Services (ECS) and EKS [or fourth if you count Elastic Beanstalk] that AWS offers I was surprised that Kubernetes was not more up front and center.Love it or hate it, since being released in the summer of 2015, Elastic Container Service (ECS) is the lifeblood behind Fargate since Fargate is a Launch Type on ECS. There has been talk of EKS on Fargate but currently there are no native integrations between the two. When leveraging ECS, Task Definitions is a main concept.The definition language for resources with Fargate and for that matter ECS are Task Definitions. Creating a Task Definition JSON is pretty straight forward by filling in the Task Definition Parameters. If leveraging Fargate, there are limitations with what Task Definitions can be used which is well documented.
Task definitions can be created directly in Harness. There is certainly choice and opinion when leveraging one of several AWS’s container offerings.
Builders have choice; Pure ECS vs EKS vs Fargate
During AWS re:Invent 2018, AWS started to leverage the term “builders” pretty heavily. The internet was ablaze with debate on developer vs builder, but AWS has created a new generation of folks who expect instant infrastructure.If you have not leveraged ECS on EC2 (aka pure ECS), EKS, or Fargate before a great tactic to learn is to start with the latest and work your way backwards. Abby Fuller from AWS has a great video diving into what Fargate provides.With that as a baseline, you can see the split between ECS on EC2 and Fargate which Totalcloud’s article is great on explaining; basically ECS on EC2 has a wider control plane and less prescription that Fargate. If Kubernetes is your poison, then EKS is your option today or leveraging a Kubernetes Vendor to deploy into AWS.
Can chose inside Harness Platform which ECS type.[/caption]Working with clients early on their Fargate journey, a common concern that Fargate gets expensive pretty quickly. Like any AWS Service having an understanding how the service is priced and scaled is critical when containing cloud costs. This video by Agent of Change is the best I have seen breaking down the costs between the three services.
Sample Fargate ECS Cluster. Like any modern cloud vendor there is certainly choice. Creating pipelines to support choice is crucial for the modern organization.
Have your cake and eat it too with Harness
One of the greatest powers of the Harness Platform is flexibility. Imagine where your pipelines can support multiple models of infrastructure deployment. Harness has first class ECS support and Harness Delegates can be deployed on Fargate Launch Types.
Harness Fargate Pipeline example.[/caption]We can walk through a simple and elegant example created by one of our Sales Engineers, Greg Kroon. In this Fargate Deployment example, we take advantage of Harness’s native ability to deploy to ECS.
If you are dabbling in multiple places to run your containers, the Harness Platform can easily provide the pipeline(s) needed to deploy/roll-back confidently in Kubernetes, ECS on EC2, and Fargate. As additional cloud services and orchestrators come about, Harness will be there to provide you with the confidence your pipelines will be robust.