Overcoming GitOps Scaling Challenges with Harness

Authors:

Table of Contents

In April, leaders and thinkers in the GitOps space gathered at GitOpsCon to discuss the latest industry trends and share their insights. Attendance was high, and excitement about GitOps was palpable. It's clear that the adoption of the GitOps paradigm is growing, with new companies joining the movement every day.

The one-day conference was packed with presentations focused on implementing and scaling GitOps solutions. Growing pains were a central theme of the most attended talks. For instance, an engineer from Nasdaq discussed easing the pain of platform engineers. Other talks were case studies on scaling journeys and accelerating adoption, highlighting the headaches engineers faced. At an enterprise level, scaling GitOps becomes a massive challenge.

So how can you scale your GitOps solution?

Industry Advice: Adobe Case Study

At the conference, there was a wealth of advice and numerous experts to learn from. Adobe, for example, detailed their journey in their talk, “Scaling a GitOps Platform at Adobe.” They were building their own internal GitOps platform on top of Argo and faced similar growing pains.

Here are a few common issues they faced:

High load on GitOps engine components (e.g., CPU)
Poor GitOps engine performance due to unnecessary work
Poor GitOps engine performance due to unnecessary and frequently scheduled tasks
Poorly optimized resource usage
Excessive GitOps engine timeouts and restarts
Problematic workloads causing unnecessary load on the GitOps engine
Reaching etcd database size limits

Generally, all these issues are related to limits imposed by the agent configuration. Redefining and fine-tuning the parameters of the deployed agent allowed some of these issues to be resolved. This advice is valuable, and Harness has begun working on a way to allow users to choose a “scaled” option for the Harness GitOps Agent based on this feedback.

However, when problems are related to load, fine-tuning an agent only goes so far. Scaling this way has a hard cap, and Adobe agrees. They started seeing issues with ArgoCD after deploying 1,500 applications, a level that was quickly reached when embracing the GitOps paradigm. It became clear that they needed the ability to quickly spin up and manage an ever-increasing number of Argo instances as Adobe grew.

This problem is not unique to Adobe. Physical limitations, business requirements, or privacy concerns all necessitate multiple instances of a GitOps agent. At Harness, we've had customers who put an agent in every cluster, give one agent to each business unit, or one agent to each project to enforce isolation. These use cases arose from different internal requirements but each required multiple agents. This raises a new problem: How do you manage agents, automate their creation and deployment, and enable environment promotion in a single system that allows teams to endlessly scale their solution as needed?

With ArgoCD, for example, these issues remain unsolved as it only allows you to manage or view a single instance at a time. Figuring out which instance you're using and where you need to log in is a hassle and an expensive, time-consuming problem. Even adding a new instance or deployment can take some customers weeks.

In the end, Adobe solved this problem. They spent years working on a platform that allows customer teams to spin up their own Argo applications as needed without overloading any one agent. Their story was a success, but it was hard-fought. It took them two years, multiple outages, and a full tiger team to reach a desired level of scalability and reliability.

So what’s another option?

Grow with Harness

At Harness, we found customers facing these challenges and more, which led to the creation of Harness GitOps, our managed GitOps solution for enterprise. Our platform effectively removes most of the limitations of horizontal scalability in ArgoCD, allowing you to manage a GitOps estate of any size through a single interface. With Harness GitOps, engineering teams can view and manage all their agents in one place. Remember all those customer examples mentioned earlier? Each use case was solved by the platform. Our users were able to plug in their existing solutions (we integrate with ArgoCD and Flux) or create new ones within Harness, avoiding the management and scaling headaches others might face.

Using Harness to Manage Scale

The Harness GitOps Agent is a critical component that enables enterprise scalability and centralized management. By installing the Harness GitOps Agent alongside ArgoCD in each cluster, users can leverage the full power of Harness GitOps without the need for Argo Server. The Agent provides a direct link to Harness pipelines, allowing users to easily manage their GitOps workflows alongside their existing CI/CD processes. It also connects ArgoCD instances to the central Harness GitOps observability dashboard, providing a single pane of glass for monitoring and managing the entire GitOps estate.

Architecture of Harness Gitops

You can view all your clusters through a single pane of glass in Harness Manager.

By distributing the workload across multiple Agents, Harness GitOps ensures that the system remains performant even as the number of ArgoCD instances and applications grows. This scalability, combined with centralized management and seamless integration with existing tools, sets Harness GitOps apart from other solutions in the market.

Take a look at Ruckus, who scaled Kubernetes with Continuous Delivery and GitOps.

‍

One way this scale is achieved is through our first-class support of ApplicationSets. As a refresher, a typical GitOps application syncs a source manifest to a destination cluster. If you have multiple target clusters, you could create separate GitOps applications for each one, but that makes management more challenging. What if you want to sync an application with hundreds of target clusters? Managing hundreds of GitOps applications is not easy.

That's where ApplicationSets come in. They function as an application factory that allows you to define a single template that syncs to multiple target environments. At Harness, users can see and sync the entire AppSet at once. They can choose to synchronize the entire set if there is a change in the template or individual applications, giving the fine-grain control some users might want.

Harness PR Pipeline, Super Hero for GitOps

The benefits of Harness GitOps don’t stop there. As part of the larger Harness platform, it leverages powerful automation capabilities. While an ApplicationSet helps manage a suite of applications, individual applications created by the set still need management. Harness simplifies this by allowing you to create a PR pipeline for any application just by specifying the environment associated with your GitOps cluster. This automated PR pipeline streamlines your change process, enabling you to scale efficiently.

Moreover, since this pipeline is integrated into the Harness platform, you gain access to a comprehensive suite of features designed to enhance your delivery and deployment processes. These features include triggers, approval steps, pipeline templates, and many more options, ensuring you can build the most robust solution for your GitOps strategy.

Scale Should be Painless

Scaling when shifting to GitOps can be anxiety-inducing, but it shouldn’t be. With Harness GitOps, not only were our customers able to solve their scalability challenges, but they also did so without spending the resources it would take to make a custom solution of their own, saving them valuable time and money. Teams can now focus on more valuable tasks and accelerate their GitOps adoption rather than worrying about managing multiple ArgoCD instances.

And now? Creating a new GitOps cluster is as simple as clicking a button, allowing our customers to scale indefinitely.