Continuous Delivery Tools to Consider

Continuous Delivery can feel like the wild west of DevOps. No one uses the exact definition. Does CD stand for Continuous Delivery, Continuous Deployment, or both...does it even matter? Many tools claim to do “CD” to some degree.

To understand this space, it's helpful to review the origins of continuous delivery, current trends, and a variety of tooling approaches.

Continuous Delivery Origins

The term CI/CD can be traced back to the 2000s, when Java, Javascript, and PHP emerged. Several developers came together to write the agile manifesto, the first principal of which is "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software". The agile manifesto paved the way for the release of the first CI tools in 2001, but technological limitations of the time kept CD a pipe dream.

Still, once builds were automated with local unit testing, it was natural to look to deploy to a test environment for functional testing and from there to look to replicate those deployments in production. This led to early continuous delivery tools like AnthillPro. However, the norm was to to build custom scripts on top of continuous integration tools like Jenkins to create full deployment pipelines.

Other CI tools picked up on this, and soon, every CI tool touted its ability to create Continuous Delivery pipelines. For five years, CI was synonymous with CD, and it was generally accepted that you could manipulate your CI tool to act as a CI/CD solution. 

However, companies realized they were spending too much time scripting CD. As all of you know, wherever there are custom scripts, there’s an opportunity for a new solution. This is where CI decoupled from CD and Continuous Delivery tools entered the market. 

Continuous Delivery Fundamentals

Continuous Delivery is the process of deploying build artifacts into an environment. An example of an artifact is a docker container, and the environment could be for testing or for production. 

Continuous Delivery tools help create pipelines to standardize releases. Fundamentally, pipelines can be broken down into five stages:

• Testing and QA - These tests are normally performed in addition to tests that are run during the build process. A CD tool should allow you to incorporate your existing test automation (regression testing, etc.) into a pipeline.
• Change Management, Governance, and Approvals - These steps identify a version as appropriate for production release. Typically the tests and other risks are reviewed.
• Deployment Strategies - This is the approach taken to deploy applications to production. Examples include canary deployments, blue/green deployments, and rolling deployments. Deployment strategies are defined in the CD pipeline.
• Verification - This is the process of verifying the success of a production deployment. Monitoring tools should plug into deployment pipelines to help verify.
• Rollback - If something in the verification step reveals an issue, then a pipeline should be able to roll that deployment back. 

Some CD solutions emphasize a particular area, but all will have these basic concepts. I suggest doing more research on what CD is and why you should care.

CD Tool Categories

For the purposes of this exercise, we’ve broken down CD solutions into four categories. 

1. Open-Source - As the name suggests, this category will feature open-source tools that are “free” to set up and are managed by a community.
2. CD Platforms - These solutions built products around the entire software delivery life cycle. CD makes up a substantial portion of its value proposition.
3. Cloud Provider CD Tools - These solutions are created and maintained by the major cloud providers and are optimized to work on that specific cloud.
4. CI Tools - These are tools optimized for Continuous Integration, but can be extended to accomplish CD with enough custom scripting. 

Things we looked for in good CD tools: easy installation, intuitive user interface, enterprise-grade features, low maintenance cost, and GitOps functionality. This list is not exhaustive but should give you an overview of the most popular players in the market. 

Open-Source CD

Open-source tooling is attractive for many reasons. Popular projects have hundreds of dedicated engineers bringing diverse experiences to a world-class product. Not to mention there’s no sticker price. 

The downside of open-source is the level of expertise needed to build and maintain the solution. The maintenance effort can often outbalance the “free” nature of the products. Open-source CD tools aren’t optimized for large enterprises with granular security control needs. That being said, smaller shops might find open-source to be an attractive solution. 

Best - Argo CD

Argo CD is the cream of the open-source crop. This feature-rich solution automates Kubernetes deployments using GitOps technology and supports a wide variety of config management tools, SSO integrations, and webhook integrations. It reduces administration toil by making application definitions, configurations, and environments declarative and version-controlled. Argo CD’s stated goal is to make application deployments automated, auditable, and easy to understand. It boasts 5k stars on GitHub.

Like all open-source tools, you’ll want to consider the maintenance effort and lack of governance, but overall this is a solid option to deploy code. Argo CD struggles as you look to move beyond just deployment to full continuous delivery. Migrating through a series of environments based on the results of automated testing is largely outside of Argo CD's scope today. Perhaps it would be more accurate if it were named "Argo Deploy" rather than "Argo CD".

Check out our comparison of Argo CD vs Harness to see how we stack up. Today vendors, including Harness, wrap Argo in CD pipelines to deliver a complete GitOps solution.

Good - GoCD

GoCD is an open-source “middle child.” The solution is sponsored by ThoughtWorks and has plenty of features like pipelines as code, native Kubernetes integrations, elastic agents, JSON and YAML support, and a modern interface. There’s a solid amount of plugins available on the market, which helps it fit in with your current tool stack.

Unfortunately, it’s a noticeably newer tool that lacks advanced features found in more tenured solutions. There’s quite a bit of potential for improvement here, but at the moment your effort would be better spent somewhere else. 

Good - Spinnaker

Spinnaker is an open-source solution brought to you by Netflix. There are currently 300 open Spinnaker issues on GitHub. 

Spinnaker’s major issue is setup and maintenance effort. On average, it takes Spinnaker users 6-8 months to stand up a pipeline, and by that point, enough engineering resources have been spent to justify the purchase of a different tool.

Check out our comparison of Spinnaker vs Harness to see how we stack up.

CD Platforms

Software delivery encompasses Continuous Integration, Continuous Delivery, cloud cost management, feature management, etc. Software delivery platforms decided to provide an end-to-end platform that makes software delivery a one-stop-shop. 

Each company has approached this in a different way, and therefore, each company has had varying results.

Best - Harness

Harness is a modern CI/CD platform that offers Continuous Integration, Continuous Deployment, Cloud Cost Management, Continuous Verification, and Continuous Features. Harness was founded on Continuous Delivery roots and that still remains one of the company’s cutting-edge products. It leads the pack when it comes to governance with fine-grained RBAC, full audit trails, and proprietary integrated secrets. It’s also worthy to note the incredibly easy installation, which reduces time to value considerably.

The great thing about Harness is its à la carte model. Teams are free to pick Harness modules without purchasing the entire platform. Every Harness product is developed to be standalone in quality and functionality.

Request a demo of our continuous delivery platform today.

Good - GitLab

GitLab offers a thorough software delivery platform that includes a solid Continuous Delivery tool. All the CD fundamentals exist here, but the tool lacks advanced governance features such as automatic verification, and automatic rollbacks.

The hardest part about using GitLab is its all-or-nothing model. If you want to use any of the GitLab tools, you need to use all the GitLab tools. So if you work in an opinionated tooling organization, you’ll have to convince them to leave behind some of the tools they already use. Our advice for GitLab? Stick to Git.

Check out our comparison of GitLab vs Harness to see how we stack up.

Proceed with Caution - Digital.ai 

Digital.ai attempts to be a DevOps swiss army knife. Instead, it resembles a butter knife. The company acquired XebiaLabs in 2020 in an apparent attempt to provide Continuous Delivery, but it struggles to keep up with a modern Kubernetes world.

A quick glance at the product section of their website will show you the breadth of their offerings. We recommend going with a platform that emphasizes software delivery specifically. 

Check out our comparison of Digital.ai vs Harness to see how we stack up.

Cloud Provider CD Tools

Is it any surprise that the three biggest tech companies in the world have solutions for Continuous Delivery? Amazon, Google, and Microsoft each created their own flavor of CI/CD. Something to note is these tools are optimized to work with their corresponding cloud environment. Therefore, these tools are a viable option for organizations running a single cloud environment.

Best - Azure DevOps

Azure DevOps includes CI/CD pipelines (Azure Pipelines) as well as several other DevOps modules. As expected from Microsoft, Azure DevOps offers fine-grained RBAC and audit trails, making it a smart solution for companies in highly-regulated industries. Azure DevOps also facilitates deployment verification and failure rollback.

Azure DevOps is not strictly limited to Azure environments. This is a tool that you can use across your application stack. 

Check out our comparison of Azure DevOps vs Harness to see how we stack up.

Good - Google Cloud Build

Cloud Build, Google Cloud’s CI/CD platform, was named a Leader in The Forrester Wave™: Cloud-Native Continuous Integration Tools, Q3 2019, receiving top scores in the current offering and strategy categories. This CI/CD platform provides standard pipeline features and works well with a modern tool stack. 

Honestly, this tool is solid. But it lacks the intuitive design of Azure DevOps. This is still a solid deployment option if you’re on GCP.

Proceed with Caution - AWS CodeBuild/CodeCommit

CodeBuild is a fully-managed CI/CD tool. It primarily focuses on Continuous Integration, but does offer some basic deployment strategies. CodeBuild exclusively works on AWS environments, making it a no-go for other cloud users.

Amazon’s deployment tool is confusing to set up and manage. Save yourself some time and choose something else. 

Tools that aren’t actually CD tools…

CD is a new buzzword, and everyone wants in on the action. Unfortunately, this is making it harder to distinguish between CI tools and CD tools. The reality is most CI tools can be extended to CD, but that involves custom scripting, script maintenance, and a less than ideal end solution. We think these CI tools deserve respect for their integration capability. But we also think CI and CD are separate disciplines that have different value propositions.

Best - CircleCI

CircleCI was built for speed and scalability. Build configurations are stored in your application repo as YAML, making it easier to track changes and contribute to your testing process. The Orb system allows you to extend pipelines using third-party functionality with minimal effort. These features create a standardizable CI solution that can easily be adopted. CircleCI definitely rates as one of the better CI tools currently available.

Check out our comparison of CircleCI vs Drone to see how we stack up.

Good - Travis CI

Travis CI was the first CI-as-a-service solution. It was designed for open-source projects: testing them is, and always will be, free - the creators of Travis CI offered this as a way to give back to the community. Additionally, the tool offers a streamlined experience that is easy to set up and use. Configurations are created using YAML. Travis CI lacks many of the enterprise-grade features of other CI tools (think: security and governance), but that, in turn, means it requires little effort to maintain. 

Check out our comparison of Travis CI vs Drone to see how we stack up.

Proceed with Caution - Jenkins

Jenkins is old. It was created in a different technological decade and built to solve Continuous Integration. I want to give credit where credit is due. Jenkins out-competed Bamboo (by Atlassian), TeamCity (by Jetbrains), and other CI tools of the early 2010s to become the industry standard DevOps solution. But industry standards have evolved. Any solution that requires dedicated hosting servers and 3-5 full-time engineers to manage it will have a hard time standing up to modern tools.

Check out our comparison of Jenkins vs Harness to see how we stack up.

How to Pick a Category

Each category of Continuous Delivery tools could fit a specific need. Open-source tools work well for smaller engineering teams that don’t need robust governance or a new tool bill. Platform solutions help transform the entire software delivery process and are great tools for enterprises concerned with security. Cloud deployment tools are a great option for companies hosted in one cloud. And the CI tools... well, the CI tools are great to build the artifacts deployed by your CD solution. 

Complimentary Tools

Continuous Delivery tools work best when existing tools are plugged into new Continuous Delivery pipelines. For instance:

By connecting your code repository (GitHub, Bitbucket, etc.) to your CD pipeline, you can automatically trigger your pipeline every time a new artifact is committed. 

By integrating terraform or ansible into your pipeline, you can create an automated infrastructure-as-code process. This basically replaces the functionality of tools like Chef and Puppet.

By integrating APM and Log tools into CD pipelines, platforms like Harness can use machine learning to verify the success of your deployments. 

The more of your existing toolset you can integrate into a CD solution, the more sophisticated your pipelines will be. 

Define Your Needs

At the end of the day, your organization will have specific Continuous Delivery needs, and you’ll need to decide which Continuous Delivery tool best solves those problems. Some key questions you should ask before choosing a solution:

• How much time do I have to get a delivery solution up and running?
• How much effort can we put into maintaining a solution?
• Is there a migration planned in the future?
• Do we have the expertise to script things ourselves?

There are definitely tools I recommend avoiding for your own personal sanity, but as an employee of Harness, I have an implicit bias towards the Harness software delivery platform. I recommend taking a look at Harness even if it’s just to create a baseline you can compare to other tools. Try it free today.