What Is GitOps? Learn About Benefits, Challenges, and More
GitOps: the philosophy that deployments, changes, and more should be as simple as making a commit to Git. Learn all about GitOps - and how Harness can help.
Chances are you’ve landed here after asking your search engine of choice “What is GitOps?” In this writeup, we will cover what GitOps is, what benefits an organization gains from GitOps, and how to scale GitOps for the enterprise. Each of these has sections with headers, so if you have a specific interest, feel free to jump down to the relevant section.
GitOps is a term attributed to WeaveWorks in 2017, for the philosophy that deployments, changes, and more should be as simple as making a commit to Git. Whether it’s adding a firewall rule, defining a VPC, fixing a UI bug - all of it should come from the control plane of source control. Why would you want to do this? Start with our true horror story below, and then we will explain to you exactly how you can solve that with GitOps.
Life Before GitOps
In the not too distant past, the author of this article owned deployments for an ecommerce application. The process involved a deployment pipeline, but also many manual processes, and requests to various internal service providers. Every six weeks, new changes were deployed, concluding a process that spanned months when you include the software development process. Something as simple as a few application code changes, or modifications to configuration files, were a large production. At the first sign of trouble, it was a requirement to roll back and conduct another six week test cycle.
That six week run-up consisted of deploying, testing, and retesting in each pre-production environment. If a change required firewall updates, additional server resources, or anything outside of the scope of deploying the codebase, that required a minimum two week leadup for each environment—often longer. No change could be initiated for production prior to testing in every lower environment, and infrastructure changes across multiple environments could not be completed in a single six week cycle.
This isn’t an unusual story; it’s actually very common. Companies are implementing GitOps because of all the problems listed above. The pace of changes is slowed to a crawl because of fragile, unrepeatable, non-scalable approaches to software development and deployments. GitOps is an approach for companies looking to simplify deployments of cloud-native applications.
Benefits of the GitOps Approach
Single Source of Truth
Anyone who has had to onboard into an engineering organization is likely very familiar with the headache of dealing with multiple authoritative systems. One system for firewalls, another system for DNS, a few more systems for provisioning compute if you’re on-prem, as well as in the cloud, a version control system, and finally, a CI/CD pipeline tool. Unless your organization has an exceptional cultural commitment to documenting everything, it’s as much a job to know where to find the answer as it is to parse it.
GitOps allows developers to have their version control system as the authoritative to look for answers about every configuration they care about. The benefits of this start with streamlining onboarding, providing an enhanced developer experience, all the way up to avoiding the dreaded “context switching” that cuts into productive time.
The changes made to every aspect of the application live in one place, and are versioned as each change is proposed by pull requests and implemented. With everything in code, any change can be reviewed and there is a full paper trail, and the ability to roll back changes.
Democratizing Infrastructure Management
Historically, every system has a different set of gatekeepers. Network teams, security teams, ops teams, and more. In the pre-GitOps world, achieving Continuous Deployment automation was at the mercy of at least a half dozen different groups—and whatever red tape they put up. Any update required a long request process to the owner of a given system, with carefully documented changes.
Developers provisioning cloud resources was the first major deviation from how companies manage infrastructure. Then came infrastructure as code, and ultimately GitOps, to close the loop from idea to implementation.
Standardizing/Ease of Use/Simplifying
Part of the reason for change management and gatekeeping has been to assure that what ends up in a production environment maps to lower environments. With many layers of applications, configurations, and infrastructure, standing up a new environment has often been a combination of checking many sources of truth, change tickets, and ultimately trial and error.
With all infrastructure, configuration, and code defined in a Git repository (or VC of choice), the ability to build out a new environment or port configurations across environments isn’t a multi-spint project.
Organizations are increasingly standardizing on metrics to measure the effectiveness of software delivery, with the DORA metrics being one of the most popular. A key metric measured is lead time, or in practical terms, how long it takes from a code commit to that change arriving in production. The GitOps approach shortens the Continuous Delivery cycle time to the minimum required to push new changes and features.
Shorter Feedback Loop
Velocity provides two benefits in one. Getting features to your end users faster is valuable in and of itself. Then, add on top the quick feedback on changes development teams get. It’s not uncommon that the time from pull request to user feedback is expressed in months. Revisiting, refining, and pushing changes long after a development team has moved onto another project is more time-consuming and less efficient, especially when there are multiple iterations.
What Is GitOps?
Building on top of our statement that the foundation of GitOps is all changes to an application being made as a merge request, let’s expand on what this model looks like in practice.
The Entire System is Described Declaratively
GitOps focuses on the what instead of the how. Kubernetes is the most popular platform at the moment where all application components are descriptions, not directions on how to accomplish building them out. In the declarative paradigm, a developer describes their desired state and the system they are interacting with determines when, how, and where to place applications in a way that meets the requirements described.
The key benefits of declarative infrastructure and declarative configuration are they allow software development teams to focus on their application first, not the logistics of deployment and runtimes.
The State of the System Lives in Version Control
In GitOps, the Git repository serves as the authority on the desired state of the application. Every change or rollback is funneled through Git pull requests, Git reverts, and actions oriented around the version control system.
Automatically Apply Approved Changes
Within GitOps, the process to apply changes is driven entirely from the Git repository. After working on a feature branch, a developer submits a pull request, any applicable Continuous Integration pipelines run, and the change is merged once the required process is fully satisfied. On merge, with the main branch as the source of truth, the deployment process is run by the GitOps operator in the deployed infrastructure, with no additional steps required post-merge.
Much of the chatter around GitOps has been on the push-based side of things. As a change is made, it is applied to the cluster, and regardless of the mechanics of how this is accomplished, it centers around developer changes being deployed. The other half of GitOps is the pull action, where an application is self-healing, and corrects itself to align to desired state.
Kubernetes-based infrastructure already restarts failed application instances (pods) and aligns to desired state, as mentioned previously. GitOps extends this principle to every aspect of the application. When a misalignment is spotted from the application side in configuration, images deployed, etc., the GitOps operator treats source control as the final authority and aligns the running state to the state in Git.
GitOps for the Enterprise
At Harness, we recently added our own Enterprise GitOps to the ecosystem of GitOps solutions. As we’ve seen our customers adopt GitOps, we’ve noticed a large number of challenges as we’ve worked with them to implement GitOps principles at scale.
Promoting Releases Across Environments
With multiple environments, describing desired state for the inevitable variations between test, QA, and production environments becomes increasingly difficult without implementing strategies that can be difficult to manage, or potentially a bad state. As of this writing, practitioners are split among a few possible solutions, including using a branch per environment—but this introduces a litany of potential issues, not the least of which is code not being properly merged into all branches in the correct order.
The alternative in play is a branch per environment, but go ahead and do the math on how many repositories you end up having across many environments and applications. The result of this is organizations hesitating to adopt, adopting only for select environments, or using disparate or bad practices that become a form of technical debt.
Argo CD was the Harness choice for GitOps adoption as it provided the ability to use an “app of apps” pattern, resolving at least a portion of handling this issue at scale. With a parent app in play, cascading changes across multiple applications becomes less of a barrier to entry. From there, support for the “Application set” is coming soon to resolve handling the scaling of deployments across environments.
With GitOps, the Git log provides a definitive record of all changes. But as an audit trail, it can become a burden to search for changes and tie them directly to business or regulatory implications. Within Git, all commits are recorded, but the implication of each commit takes time and effort to unearth—especially for non-technical audiences.
GitOps with Harness provides a full audit log of all the deployments and changes, tracked to allow operational and business level analysis, without requiring digging through Git commits. Searching through Git pull requests and files down to the code level is still an option on the table, but it is no longer the required starting point.
GitOps Covers One Part of the SDLC
A well understood limitation of the GitOps approach by itself is, there are many portions of the software development life cycle not covered, as it’s not a one-size-fits-all solution or magic bullet. Compiling code, running unit testing, integration testing, security scanning, and more require more tooling and processes to support the Continuous Deployment model of GitOps.
The Harness platform provides built-in GitOps as part of a holistic solution covering everything in the Continuous Integration and Continuous Deployment life cycle. A distinction is necessary here as GitOps follows an idea of pushing everything live immediately.
In addition to GitOps capabilities, Harness provides a full delivery pipeline around your process including approvals, controlled feature releases, canary / blue-green rollout strategies, and Continuous Verification. Another topic for another time, but as with every trendy technology/philosophy, the trend of the day should not be applied to every application. Sometimes, old-fashioned automated pipelines with built-in verification can solve your problem as well as an approach like GitOps.
With many applications and environments comes a set of challenges with scale. With the number of Git repositories skyrocketing, it quickly becomes hard to track environments and configurations. Centralizing vs. true autonomy quickly becomes a real problem in that applying company-wide changes or policy becomes a burden when giving teams autonomy, contrasted with the tradeoffs of centralizing configurations and creating a large bottleneck. With the race for talent on and the necessity of a developer-centric experience, setting up GitOps at scale can become a liability.
With GitOps as a very developer-centric, decentralized approach to delivery, the burden on DevOps teams is to provide the necessary tools without everything turning into the wild west. As with the benefits of a platform providing for other parts of the SDLC, Harness also provides a centralized pane of glass for managing GitOps while providing the needed autonomy to development teams.
Several key challenges the platform looks to solve are providing project-level spaces, robust RBAC, and policy powered by OPA. Combining the ability to segment spaces, provide proper access, and finally govern by way of policy described as code allows DevOps teams to fully control day-to-day operations without being overly hands-on or prescriptive in how teams build their software.
Looking to adopt GitOps in your organization and want to learn more? Check out our webinar where we dove deep into GitOps, how to adopt, potential pitfalls, and how to extend GitOps to the next level.
Wherever you are on your GitOps journey, from exploration to seasoned professional, we hope this was helpful for you. If you’d like to see GitOps in action with Harness, come talk to us and we’ll share our thoughts on opportunities with GitOps, as well as how we’re tackling the challenges.