What Is a CI/CD Platform and Why Should I Care?

To put it succinctly, a CI/CD platform is a comprehensive set of tools that help your developers, engineers, and DevOps practitioners package and deliver software to the end users. If you don’t adopt a CI/CD platform and best practices, you (and your company) are on the fast track to complete annihilation. Your competitors will be able to out maneuver you with agility, efficiency, and governance. This post will help you understand the basics of CI/CD as well as provide some examples of popular tools.

Why Should I Care About a CI/CD Platform?

Who would care about CI/CD if it didn’t positively impact their everyday work life? Depending on the role, a CI/CD platform has different benefits.

Developer / Software Engineer: While it can be fun to write a bunch of CI/CD scripts to manage the build and delivery of software, it’s no fun at all to maintain those scripts. As the scope gets bigger, maintaining tens, hundreds, or thousands of scripts becomes a source of significant toil. A CI/CD platform does away with the majority of that toil, freeing your time to do what you and your company love most - creating business functionality.

Here are a few more reasons to care about using a CI/CD platform:

• On-demand deployments
• Auto-verification & rollback
• Reduce change failure rate
• Increase quality of software

DevOps Practitioner: As a DevOps practitioner, you’re probably responsible for ensuring all software build and delivery pipelines are available, self-service, performant, and functioning as expected. You may also be responsible for the health and performance of applications in production, as is often the case for an SRE. A good CI/CD platform should make your work life significantly easier due to:

• Empowering engineers with self-service capabilities
• Reduced fragility of build and deploy pipelines
• Easier pipeline maintenance
• Standardization across all pipelines
• Faster on-boarding of new services
• Faster identification of issues during build and deployment
• Automatic rollback of bad deployments

Security/Audit: There’s an equilibrium between enabling teams to go fast and ensuring the security of the enterprise. The traditional methods of managing CI/CD pipelines come with minimal security and audit capabilities. Using a CI/CD platform with granular RBAC and a full audit trail will ensure a proper balance between security and agility.

Management: Do more with less, for cheaper. It’s a difficult mandate, but it’s also achievable using a modern CI/CD platform. Using standardized build and deployment pipelines your teams will:

• Onboard new services faster
• Spend less time maintaining scripts
• Build more business functionality
• Deploy new software on-demand
• Release fewer defects to production
• Save money while doing so

CI/CD Defined

CI is the acronym for Continuous Integration. Continuous Integration is the process that kicks off after a developer checks in code to the code repository (repo). Developers don’t work with the entire source code of a project at once, they work on a portion of the source code that they checkout from the repo. 

Great, but what’s a code repository? A code repository is the place where all of the source code files are kept. Popular code repositories are GitHub, GitLab, and Bitbucket.

Back to the CI discussion … After they finish making changes, Developers check in the code so that the updates can be merged into the master source code. The merge should in turn kick off a build process, which creates artifacts that can be run and tested. Ideally, the new build is automatically tested and feedback is provided to the developer so they know if the changes broke anything or not. This process is repeated over and over as all developers work on code for any application or service.

CD is an acronym with 2 different meanings. CD stands for Continuous Delivery. CD also stands for Continuous Deployment.

What is Continuous Delivery? Continuous Delivery is the process of packaging up your software artifacts (the ones created during Continuous Integration) and getting them to the state where they are ready to be delivered into an environment like QA, UAT, or Production.

What is Continuous Deployment? Continuous Deployment is the process of deploying the packaged software (from the Continuous Delivery process) into one or more environments without human intervention (e.g. approvals) and is often triggered by a commit or new build.

What Is the Difference Between CI and CD?

CI (Continuous Integration) is a process used by developers to create artifacts that are ready to be built into usable software. CD (Continuous Delivery) is the process of making those artifacts ready for deployment to customers in production. CD (Continuous Deployment) is the process of deploying a software build (artifact) into an environment for testing or usage.

For the rest of this article, we’ll consider CD to encompass both Continuous Delivery and Continuous Deployment.

What Is the Difference Between Continuous Delivery and Continuous Deployment?

It’s pretty simple really: Continuous Delivery prepares artifacts for deployment where humans decide if/when/where to deploy. Continuous Deployment automates the deployment of software artifacts without the need for human intervention.

The Benefits of Continuous Integration

The main point of adopting Continuous Integration is to speed up the process of merging and verifying code updates. Continuous Integration typically incorporates unit tests to see if the modified portion of the code works as expected in isolation. After unit testing, integration testing is performed to ensure that the updated code functions as expected in the context of the larger software build. Some common CI tools include Jenkins, Drone, GitLab, and BitBucket.

To summarize, here are the main benefits of CI:

• Faster developer iteration
• Better release quality
• Increased developer efficiency
• More consistent build process
• Lower risk of poor user experience

The Benefits of Continuous Delivery and Deployment

Fewer companies have been successful in achieving Continuous Delivery and Continuous Deployment than have achieved successful Continuous Integration. CI is a more mature discipline than CD. It’s important to note, however, that there are a wealth of benefits derived from implementing CD.

CD is where the rubber meets the road for software delivery organizations. It’s where all of the hard work of building and maintaining software gets delivered into the eager hands of the end users.

The process of deploying software is prone to errors and mistakes. CD has traditionally been a process controlled by a series of shell scripts. Often, there are 15-20 different shell scripts that are manually created and maintained by many different engineers. When an update is required, it can be an excessive time suck to update and ensure there are no errors. Human errors during the deployment process are responsible for issues like:

• Hey, the production server is using a test database
• Whoops, the application server ran out of memory because we used the wrong config
• Uh oh, we just deployed that new service to the wrong Kubernetes cluster causing poor performance

To summarize, the main benefits of CD are:

• Speed - automated deployments are faster than manual
• Consistency - no chance of overlooking something important
• Lower risk of a security incident by misconfiguration
• Lower cost / Improved developer efficiency
• Faster MTTR (Mean Time To Repair) using automated rollback

The Components of a CI/CD Platform

A well-rounded CI/CD platform enables an automated software delivery process from code check-in to post-deployment verification and beyond.

Diving into specifics, a basic CI/CD platform will need the following modules:

• Continuous Integration - Decrease lead time for new builds
• Continuous Delivery - Always have a functional new release ready to deploy
• Continuous Deployment - Deploy new release at will

Those are the basics that will make it possible to build, package, and deploy new software releases. While the basics are a good starting point, there is more to consider when building out automated software delivery pipelines.

Important considerations beyond the basic capabilities listed above:

• How fragile are the pipelines?
• How long does it take to build a pipeline and onboard a new service?
• Is it easy to make a change or is the maintenance of the pipelines more work than not having them at all?
• Who can execute the pipelines?
• Is there an audit trail for regulatory or other audits?
• What was the cost of each change or new service? Is the typical over-provisioning occurring and wasting profits?
• How quickly are bad deployments identified and resolved?

Taking the above considerations into account, a CI/CD platform should include the following:

• Continuous Integration - Decrease lead time for new builds
• Continuous Delivery - Always have a functional new release ready to deploy
• Continuous Deployment - Deploy or roll back new release at will
• Continuous Verification - Immediately know the health of a new release
• Cloud Cost Management - Understand the financial impact of every release
• Continuous Governance - Implement granular access control and know who performs every pipeline action

How a CI/CD Platform Enables More Frequent Code Deployments

Not every company needs to deploy multiple times per hour, but every company needs to be able to deploy a new release on-demand, in minutes, with confidence that that the new release is not negatively impacting customer satisfaction.

A CI/CD platform is designed to help you automate the steps between a developer checking in their code and the release of that code into production. Without a CI/CD platform, DevOps practitioners typically string together shell scripts that perform all of the necessary steps. These shell scripts have weak, if any, governance and are sensitive to changes making them difficult to manage.

CI/CD platforms simplify and standardize the pipeline creation and management processes so much that new pipelines can be created in hours and maintenance takes only minutes. CI/CD platforms offer reusable templates and a self-service model, allowing software engineers to build their own pipelines and deploy on their own. A well rounded CI/CD platform also incorporates strong governance so you can control who has access and audit all the actions performed.

Improving the Quality of Software Deployments Using a CI/CD platform

Testing, testing, and more testing. Testing must occur early and often throughout the software delivery lifecycle.

CI pipelines should execute unit tests, integration tests, OSS Security tests, dependency tests, and license tests.

CD pipelines should execute smoke tests, regression tests, vulnerability tests, browser tests, load tests, performance tests, availability tests, quality tests, etc.

The testing doesn’t stop once the software has been deployed. It’s absolutely vital to monitor the software release to understand the quality. A robust CI/CD platform will integrate with all major monitoring solutions and interpret the data to verify the quality of the deployment. If major issues are identified then the CI/CD platform should recommend or initiate the appropriate action, like a rollback.

The Importance of Governance in the Software Delivery Process

Often overlooked, governance is one of the most important considerations of a CI/CD platform. Automation introduces a new risk, the risk that anyone can easily disrupt the digital business by kicking off a pipeline without authorization or knowing what they are really doing. Whether the automation is done via scripts or through a more robust software platform, the problem remains the same. Governance is much easier to implement using a software platform than it is using scripts. A good CI/CD platform will have granular role-based access control (RBAC) that can control the authentication and authorizations of each user. It’s imperative to closely analyze the RBAC capabilities of any CI/CD platform in this time of highly publicized security breaches.

Another important governance capability is integrated secrets management. It is an anti-pattern to keep secrets in plain text in any configuration files. Many companies have begun using secrets managers like AWS Secrets Manager or Hashicorp Vault. CI/CD platforms also can contain their own proprietary secrets manager if you don’t have one already.

Last but not least, audits (that dreaded word) happen at the worst times. There is never really a good time for an audit, but your CI/CD platform can make your next audit faster and easier if it tracks and records all major activities, when they were performed, and who performed them.

Governance is a challenge for many CI/CD platforms. We've created another blog post that explores this and other CI/CD challenges in more detail.

Simplifying Canary Deployments and Blue Green Deployments using a CI/CD platform

Blue/Green and Canary deployments can be difficult to implement. They are advanced deployment patterns and require certain expertise to successfully achieve.

Blue/Green Deployment: The Blue Green deployment pattern relies on 2 separate production environments that are as similar as possible (one labeled as Blue, the other as Green). When it’s time to update software you deploy the new version to the Green environment and then instruct your routers/switches to send all traffic to the newly updated Green environment. If anything goes wrong you can nearly instantly roll back by telling your routers/switches to revert to sending all traffic back to the Blue environment which has the previous software version already running.

Canary Deployment: The Canary deployment pattern uses a single production environment. When it’s time to update software you deploy the new version to a subset of the total nodes and instruct the routers/switches to send a small portion of traffic to the updated nodes, while the majority of traffic continues to use the old version. Assuming all goes well with the traffic running on the updated nodes, you gradually roll the update to more nodes while sending more traffic to the updated services. Over time you either update all nodes to the new version or roll back to the previous version.

These deployment patterns are perfect candidates for automation via a CI/CD platform. Since these are well known patterns, the knowledge of how to execute these deployments is codified and made repeatable within the platform. This removes the burden of expertise from the developer or DevOps practitioner and places the burden on the company producing the CI/CD platform. There is no need to reinvent the wheel, simply re-use what has already been created.

Releasing Better Software Using Continuous Verification

When you consider all of the information that a full CI/CD platform has access to, it becomes apparent that CI/CD is becoming the center of the software universe. CI/CD platforms have access to the following data:

1. Change Events
2. Service Availability
3. Resource Allocation and Consumption
4. Service Quality
5. Transactional Performance
6. Security Details
7. Cloud Cost

This is an exceptionally rich data set for anyone who is responsible for ensuring the smooth operation of software and business continuity in production. By applying ML (Machine Learning) algorithms to this rich data set, a CI/CD platform can, and should, provide actionable insights which lead to the following benefits:

• More stable releases
• Fewer production defects
• Faster identification of customer impacting issues
• Less developer toil triaging incidents
• Reduced cloud spend

How Fast Should My CI/CD Pipelines Be?

DORA (DevOps Research & Assessment) publishes an annual report based on a survey sent to DevOps practitioners. This report contains benchmark information that is useful for comparing the maturity of your own DevOps capabilities.

The State of DevOps report lists 4 key metrics that every organization should track:

1. Deployment Frequency - “How often does your organization deploy code?”
2. Lead Time for Changes -  “How long does it take to go from code commit to code successfully running in production?”
3. Mean Time to Recover (MTTR) - “How long does it generally take to restore service when a service incident occurs?”
4. Change Failure Rate - “What percentage of the changes either result in degraded service or subsequently require remediation?”

The report also breaks down maturity into 3 categories (Low Performers, Medium Performs, High Performers) with metric ranges to assist in identifying which category you fit into. Here are the breakdowns:

1. Deployment Frequency
2. Low Performers: Between once per month and once every 6 months
3. Medium Performers: Between once per week and once per month
4. High Performers: On demand (multiple deploys per day)
5. Lead Time
6. Low Performers: Between one month and 6 months
7. Medium Performers: Between one week and one month
8. High Performers: Less than one hour
9. MTTR
10. Low Performers: Less than one day
11. Medium Performers: Less than one day
12. High Performers: Less than one hour
13. Failure Rate
14. Low Performers: 31-45%
15. Medium Performers: 16-30%
16. High Performers: 0-15%

As you can see from the metrics above, becoming a high performer takes some real effort. It drives the need for a simple yet powerful CI/CD platform but it also requires an investment in people and process.

What’s the impact of being a high performer? It’s summarized in the 2017 State of DevOps report:

“We found that high performers were more than twice as likely to achieve or exceed the following objectives:

• Quantity of products or services.
• Operating efficiency.
• Customer satisfaction.
• Quality of products or services provided.
• Achieving organizational and mission goals.
• Measures that demonstrate to external parties whether or not the organization is achieving intended results.”

Popular CI/CD Platform Solutions

Harness - Harness is a modern CI/CD platform that currently offers Continuous Integration, Continuous Deployment, automated deployment verification and rollback based on machine learning, Feature Flags, fine-grained RBAC, full audit trails, proprietary and integrated secrets management, and cloud cost monitoring with suggestions. It was built for cloud native technologies but it also works well with legacy technology.

Harness is available as a SaaS platform but also has an on-prem offering for those companies who require it. Harness is a commercial software vendor but also offers an open-source version of its Continuous Integration module.

Jenkins - Jenkins has been around for many years and, as a result, is geared towards legacy technologies but also supports newer cloud native architectures. Jenkins offers Continuous Integration, Continuous Delivery, scripted deployment verification and rollback, coarse-grained RBAC, audit trail via plugin, and integrated secrets management.

Jenkins is open-source but is also commercially available from Cloudbees. At this time, Jenkins is only available on-prem, no SaaS offering.

GitLab - GitLab is a bit different from the other CI/CD platforms listed here since it is also a popular code repository. GitLab offers Continuous Integration, Continuous Deployment, manual deployment verification and rollback, fine grained RBAC, full audit trails, and proprietary secrets management.

It’s important to note that the GitLab CI/CD platform will only work with the GitLab code repository and the proprietary secrets manager, unlike the rest of the platforms listed here.

GitLab is available both on-prem and SaaS.

Azure DevOps - Azure DevOps is a CI/CD platform from Microsoft. This platform has may different modules consisting of a code repository (Azure Repo), CI/CD pipelines (Azure Pipelines), artifact management (Azure Artifacts), community extensions, test tools (Azure Test Plans), and Kanban boards (Azure Boards). As expected from Microsoft products, Azure DevOps offers fine-grained RBAC, audit trails, and deployment verification with rollback. Azure DevOps works well with cloud native technologies and also has support for legacy technologies, but legacy support falls mostly under the Microsoft software umbrella.

Azure DevOps is offered as a SaaS solution but also has an on-prem version called Azure DevOps Server.

Going Deeper With CI/CD Platforms

This article was intended to provide a high level overview of what CI/CD platforms have to offer. As a next step, I suggest reading the CI/CD Buyer's Guide, which provides in-depth comparisons of CI/CD tools. It should help you make an informed decision on which tool to go with.