GitLab and Harness From Someone Who's Been There, Done That

In my last role, I was one of the technical architects for a field engineering organization. As a group, we were charged with tackling a variety of problems, including custom services, building integrations, contributing to open source projects, and “creative” solutions. One of the key problems I was tasked with solving as we scaled up was CI/CD, including implementation, maintenance, standards, and tying into existing Jira processes. 

Challenges faced:

• Team that grew from 3 to 15+ in less than two years.
• 50+ projects of varying complexity and scope and many with support SLAs attached.
• Distributed team in Vietnam, Canada, and the US.
• Many different languages in use (Python, Golang, React/JS, etc).
• Many distribution and deployment methods.
• Mixture of legacy and modern deployment infrastructure involved in testing and deployment.

GitLab was a recent choice for the broader engineering organization, moving from a plain git implementation, so it was de facto our system of choice. Prior to GitLab, the pipelines we had in place consisted of some crude Jenkins implementations that inspired many an angry Slack tirade. 

The Good

The basic onboarding of projects was very easy. Given that you have infrastructure (runners) configured upfront, you simply needed to add a .gitlab-ci file with a few simple directions to a new project, and you were able to start running some simple pipelines with five minutes worth of effort. Auto DevOps templates worked well as a starting point for building out more complex pipelines.

The integration between CI and git was well done, which was very useful for enforcing good git process and hygiene. Merge requests with pipelines adjoining and Jira issues linked directly provided solid collaboration, review, and governance.

The Bad

Reusable components in the form of “includes” were a useful construct, but scaling this approach was difficult when many projects had many “includes.” Getting a holistic view of the pipeline was difficult from the developer’s point of view. The first time I got hands-on with Harness, I immediately appreciated that templating is straightforward, and easy to visualize.

More advanced patterns and optimizing processes was, more often than not, a trial and error exercise. The self-service aspect of GitLab is good, in that you don’t need a dedicated DevOps person to build out every pipeline, but the downside is that solving more complex problems requires a substantial time investment to build, test, and validate solutions before rolling them out to the broader set of projects. In Harness, the ability to develop, visualize, and test components (workflows) both in YAML and in the UI helps streamline the process for building out more complex logic and processes. 

The Jira integration (to be able to tie merge requests to issues and vice versa) worked well for some use cases, but in the context of CI/CD pipelines, it couldn’t really tie into our desired process without extra scripting and integration. Pipelines could not directly update and add context to Jira. Given that Jira was our final source of truth for releases, this process ended up with some manual steps instead of a closed loop. Jira integration with Harness allows for tickets to be opened, updated, used for change management, and more - directly from pipelines.

The Ugly

Deployments presented a variety of challenges for our team. Auto DevOps worked for a few specific projects, but most required customization and included legacy infrastructure. In the end, most deployments were done manually, or by a combination of scripting and manual work. This is a common challenge we see at Harness, where a lot of customers attempt CD by way of extending CI tools, only to realize they should have gone with a true CD tool from the start. The challenges in testing and maintenance from using incorrect tooling simply weren’t worth it. 

There were common tasks we performed, such as pushing to a Docker registry or provisioning resources where some solutions were two liners, while others were significantly more complicated and required enough variation where inline scripts did not scale. The solution to most of these problems was initially to script out the use cases, but across many repos, we found this lacking and sharing among projects in a scalable way became a challenge. Configurable connectors to common services provides a way to make these actions available from the top down in a secure, repeatable way.

Provisioning and configuration was a challenge in itself. Being able to consistently provision, validate, and configure new VMs and infrastructure came down to manual scripting, and some manual processes. Harness provides the ability to use Terraform and CloudFormation in pipelines.

Assuring that new projects met standards without manual review was difficult, given the large volume of projects and limited resources. This was addressed with a combination of auditing, project reviews, and templating. In cases of more homogenous projects, this was much easier. For example, a subset of our projects followed a standard format in language and structure, so enforcing good practices was much more streamlined. For the varied, one-off, and oddball projects, this resulted in a lot of toil. Harness provides pipeline governance out-of-the-box, giving the ability to manage standards for pipelines from the top down.

Finally, GitLab lacked a global view for build and release activity. We ultimately solved this using Jira as our “single pane of glass”, but from the engineering side, we had to check in individual projects or write API scripts to gather information. Knowing the full who, what, when, where, and why required gathering information from both Jira and many projects across GitLab. Management required regular updates and information on all these activities, so in the end there was a lot of time spent compiling information. Harness provides out-of-the-box dashboards, the ability to create and share custom dashboards, and an upcoming update will create an even more robust experience with custom dashboards.

Key Takeaways

GitLab’s main benefit as a CI/CD tool is being tightly integrated with source control. This is one of the reasons Harness CI Enterprise provides the Developer Experience, where integration with source control beyond the git level enhances the overall workflow and utility of the tool.

Many of the challenges faced for my team came down to our diverse set of use cases that ultimately had to be resolved with scripting and manual toil. Deploying to diverse sets of infrastructure was one of the first real things I loved about Harness when I started working here - it wasn’t limited to Kubernetes and EC2. 

Another key challenge was common repeated tasks. Connectors for common interactions such as Jira, Terraform, and more are provided out-of-the-box with Harness.

Other features, such as visibility at the project level for developers, were very useful - but for an organization-wide view, there was a lot to be desired. Harness provides both out-of-the-box dashboards with what matters most, and the ability to create custom dashboards to get full insight into every aspect of the DevOps lifecycle.

Final note, I would be remiss to not even mention one of the coolest features in Harness: Continuous Verification. GitLab provides tie-ins to monitoring, but overall is very limited in scope. For my team, we didn’t see much value in setting it up vs. existing monitoring and processes. Harness provides the ability to tie into already-configured metrics providers and use that data to understand steady state, detect issues, and automatically roll back problematic deployments. Our customers have used CV to reduce time watching monitoring after deployment and to increase quality.

Hopefully you’ve learned a bit more about the differences in GitLab and Harness. If you’re as excited as I was - and still am - about this platform, give it a spin today!