CI/CD Pipeline: Everything You Need to Know

| Harness Blog

1. A CI/CD pipeline automates the entire journey from pull request to production: build, test, security scanning, promotion through environments, deployment, and post-deploy verification.
2. Map the types of tests to the pipeline stage — unit tests on every commit, integration tests on post-build, load tests on pre-production only, and DAST on nightly builds.
3. DORA research has historically associated elite software delivery performance with lead times under one hour, frequent production deployments, low change failure rates, and fast recovery from incidents. AI-assisted verification and auto-rollback make high deployment frequency sustainable.

Continuous Integration handles code integration, builds, and early validation. Continuous Delivery extends that automation with testing environments, artifact promotion, and controlled release into production. They are the critical path between a merged pull request and software in front of users.

The hard truth is that writing code has never been faster, but safely shipping it is still a bottleneck. The DORA report has consistently shown that while the velocity of an individual developer has increased, delivery throughput and deployment stability have not been able to keep pace. The bottleneck is not how quickly you can code. It's how quickly you can validate, promote, and roll out with certainty.

Harness CI/CD automates the entire delivery lifecycle - from building and testing code with Harness CI to deploying and verifying releases with Harness CD. AI-assisted verification catches deployment failures automatically, Test Intelligence cuts test cycles by up to 80%, and pipeline templates let platform teams standardize delivery across hundreds of services.

What Happens When You Add CI/CD?

Without a pipeline, the delivery process looks like this: developers merge infrequently because integrating branches is painful, testing is manual and happens right before release, and deployments are high-coordination events that take days to plan. Bugs show up late, when they're expensive. Every release is a bet.

With CI/CD, each commit triggers an automated build and test run. Releases ship in smaller chunks, more frequently. The failure surface shrinks because you're testing 20 lines of change instead of 2,000. When something breaks, you know exactly which commit caused it.

It's more than just speed. Teams stop firefighting and start shipping predictably.

What Is a CI/CD Pipeline?

A CI/CD pipeline is an automated series of steps that moves code from a pull request to a running deployment. These steps usually include compiling and building the application, running tests, scanning for vulnerabilities, packaging into an artifact, promoting through environments, and deploying to production.

Pipelines are code, usually YAML, that live in version control next to your application. That means a versionable, reviewable, reproducible delivery process. A pipeline run triggered by a commit in March should be reproducible in October when dependencies, base images, runner versions, secrets, and infrastructure inputs are pinned or controlled.

A pipeline can be triggered by a pull request, a merge to main, a new artifact version appearing in a registry, or a scheduled release window. Complex systems typically have multiple pipelines: one for building and testing, another for promotion through environments, and possibly a separate one for infrastructure provisioning.

What Your Pipeline Is Optimizing For

Not all CI/CD pipelines have the same objective. The structure follows the constraint you're solving for.

Moving code across environments

If your biggest pain point is reliably deploying a service across dev, staging, canary, and production, your pipeline is fundamentally an environment orchestration system. Each stage gates the next. Promotion is automatic when tests pass, and manual when a team needs to approve. Deploy to any target (cloud, on-premises, or Kubernetes) from a single pipeline definition, with no environment-specific pipeline sprawl.

Testing at the right stage

Unit tests run on every commit. Integration tests run after a successful build. Load and soak tests run before production. That progression isn't arbitrary. Running a 45-minute load test on every PR is wasteful. Skipping it entirely before a production release is a risk. Here are some best practices for test automation to help you decide what to run, where, and when.

Coordinating multiple services

When a single feature requires changes across three microservices owned by two teams, the pipeline needs to coordinate those deployments in the right order. If service B depends on the new API contract from service A, deploying them in the wrong order breaks production. Service orchestration pipelines handle dependency ordering, parallelism where safe, and rollback across the full service graph.

Post-deployment outcomes monitoring

The pipeline doesn't end at deployment. If you're tracking error rates, latency, and saturation as deployment criteria, the pipeline continues monitoring for 10, 20, or 60 minutes post-deploy. If error rate crosses +0.2% above baseline, the pipeline triggers an automatic rollback. That's an outcome-driven pipeline, not just an artifact delivery pipeline.

Developer self-service with guardrails

Platform teams usePolicy as Code to define what has to be true before anything reaches production: security scans passed, approval received, test coverage above threshold. With those guardrails in place, developers run their own pipelines without waiting for platform team intervention. Platform engineering teams use this model to standardize the 90% that should always be consistent, while giving product teams controlled flexibility for the remaining 10%.

What Goes Into a CI/CD Pipeline

Build stage

The build stage compiles source code and packages it into a deployable artifact. For a Java service, that means running Maven or Gradle to produce a JAR, then running docker build to package it into a container image. For a Node application, it means npm install, running the build script, and producing a dist folder or Docker image.

Incremental builds skip recompiling unchanged modules. Cache Intelligence automatically caches dependencies between runs. Teams using these optimizations typically see 2-4x faster build times without changing a line of application code.

Unit tests and dependency scans also belong in this stage. Run them here, early, when failure is cheap to diagnose and fix.

Infrastructure provisioning

Modern pipelines provision infrastructure within the delivery flow, not as a separate pre-step. When a staging environment needs a new database or a Kubernetes namespace, the pipeline creates it using OpenTofu or Terraform before deploying the application. Pipeline progression is gated on infrastructure provisioning success. If the Terraform apply fails, the deployment doesn't run.

This matters for environmental consistency. If the same IaC code that provisions production also provisions staging, environment drift stops being a surprise at release time.

GitOps sync

In a GitOps model, the pipeline writes the desired state to Git rather than pushing it directly to the cluster. Argo CD watches the Git state and continuously reconciles the cluster to match it. For many stateless application changes, rollback can be as simple as reverting the Git commit that changed the desired state. Stateful changes, database migrations, and external dependencies may still require additional rollback planning. If the real state differs from the declared state, automatic reconciliation occurs.

GitOps doesn't replace CI/CD pipelines. It handles deployment and reconciliation. You still need a pipeline to run builds, tests, security scans, and make promotion decisions. Harness GitOps combines both: full pipeline orchestration with GitOps-based deployment and drift detection.

Test stages

Map test types to the right pipeline stage, not the most convenient one.

• Unit tests: Every commit. Milliseconds to seconds per test. Fail fast, fail early.
• Integration tests: After a successful build and deploy to a dev or test environment. These usually need a running application or dependent services, so they often run after the build in an ephemeral, dev, or test environment.
• Contract tests: Before any service-to-service integration. Catch API breaking changes before they reach a shared environment.
• Load tests: Before production promotion, not on every PR. A realistic load test takes 10-30 minutes. Gate it at the staging-to-production transition.
• Regression tests: Post-deployment on a scheduled basis, or triggered by deployments to catch regressions that only surface under production conditions.

Test Intelligence uses ML to analyze which tests are actually relevant to the code that changed. On a large test suite, this can reduce test execution time by up to 80% without dropping coverage on the code paths that matter.

Security scanning

Security scanning belongs inside the pipeline, not outside it. The practical split for most teams: SAST on every PR, container image scanning before any environment promotion, dependency scanning (SCA) on every build, and DAST on nightly builds or before production. This early integration is the foundation of a DevSecOps practice and shift-left security.

Vulnerability scanning in your CI/CD pipeline: SAST catches issues fast but generates noise. DAST is slower and more accurate but can't run on every commit. Full scans slow pipelines, so scope them to changed code and changed containers — not the entire codebase on every push.

For supply chain security, enforce SLSA compliance by generating provenance attestations at build time and verifying artifact integrity before any deployment. Block anything you can't trace back to its source.

Release strategies

The release stage defines how the new version replaces the old one. The strategy you choose depends on your risk tolerance and infrastructure constraints.

Rolling deployment

Each instance is updated separately (or in small groups). The old version handles traffic until all instances flip. Simple to implement, no extra infrastructure cost. The tradeoff: both versions run concurrently during the rollout, so your application must be backward-compatible across versions. Read more in the rolling deployment guide.

Blue-green deployment

Two full production environments run in parallel. A load balancer shifts traffic from the stable environment (green) to the new one (blue). Rollback is instant: flip the load balancer back. The cost is real, though. Running two full production environments doubles your infrastructure spend during the transition window.

Canary deployment

The new version starts with a small traffic slice — typically 1-5% — and expands in phases based on metrics. A common progression: 5% for 30 minutes, then 25% if error rate and latency hold, then 100%. If any phase fails the success criteria, the pipeline rolls back automatically. Canary is often a strong default for production deployments when teams have the traffic routing, observability, and rollback automation needed to evaluate each phase safely. The tradeoff is validation complexity: you need clear metrics to promote, not just "no errors." For implementation details, read the full canary deployment guide.

Deployment verification

AI-assisted deployment verification compares real-time metrics against a baseline window, typically the same time period from the previous deployment. It looks for error rate increases, latency degradation above a threshold (say, p99 latency up 15%), throughput drops, or resource consumption spikes that weren't there before.

When a regression is detected, the pipeline rolls back automatically without waiting for a human to notice. That changes how you think about deployment frequency. Shipping daily or multiple times per day becomes reasonable when the system catches failures and recovers within minutes, not hours.

GitOps and CI/CD: Working Together

GitOps and CI/CD solve different parts of the delivery problem. CI/CD pipelines handle building, testing, scanning, and making release decisions. GitOps handles the state of deployed environments. See the full GitOps guide for deeper coverage.

The split in practice: your CI pipeline builds the image, runs tests, pushes the artifact to a registry, and then updates the deployment manifest in Git with the new image tag. The GitOps reconciler picks up that change and applies it to the cluster. If something drifts from the declared state, the reconciler corrects it automatically.

GitOps doesn't handle approvals, build orchestration, promotion logic, or Jira ticket updates. Your pipeline does. Argo CD best practices covers how to integrate both effectively for Kubernetes teams.

What a Well-Functioning CI/CD Pipeline Looks Like

DORA's research sets concrete benchmarks: elite-performing teams have a lead time under one hour from commit to production and a change failure rate below 15%. If your pipeline takes longer than an hour end-to-end, or more than 2 in 10 deployments fail, the pipeline design needs work.

The specific metrics to track:

• Lead time for changes: Time from commit to running in production. Target: under 1 hour for elite, under 1 day for high.
• Deployment frequency: How often you ship to production. Elite teams deploy multiple times per day.
• Change failure rate: Percentage of deployments that cause a degradation or outage. Target: below 15%.
• Mean time to recovery (MTTR): How long to restore service after a failure. Target: under 1 hour.

Pipeline analytics and insights turn these from manual calculations into live dashboards. You can see exactly where your pipeline slows down, which stages are flaky, and where change failure rate spikes.

On security: every build should run in an isolated environment. Secrets should never be in plaintext in pipeline configs. Artifact integrity (SBOM generation, provenance attestations) should be non-negotiable before anything reaches production. Compliance should be a byproduct of running the pipeline, not a separate audit process.

How CI/CD Pipelines Transform Developer Workflows

The practical impact on developer experience: developers merge to main more frequently because integration is automated, not manual. They get test results in minutes, not days. They know exactly what broke and in which commit. They can deploy to production themselves without opening a ticket to the ops team.

Platform teams use pipeline templates to codify the delivery standard. Build once, extend everywhere. Security scans, approval gates, and environment promotion logic are defined once in the template and inherited by every service that uses it.

Visualizing your DevOps data stops being a reporting exercise and starts being a diagnostic tool. You can see deployment frequency per service, identify which teams have high change failure rates, and trace a production incident back to a specific pipeline run.

Harness: Automate CI/CD

Harness covers the full delivery lifecycle: building and testing code with Harness CI, deploying and verifying releases with Harness CD, managing GitOps state, governing pipelines with policy-as-code, and surfacing engineering metrics with Software Engineering Insights.

• Slow builds: Cache Intelligence and Build Intelligence automatically optimize dependency caching and incremental compilation. Up to 4x faster builds, no manual cache configuration.
• Test suite bloat: Test Intelligence runs only the tests relevant to the changed code. Teams commonly see 50-80% reductions in test execution time.
• Deployment risk: AI-assisted deployment verification compares live metrics against baseline automatically. Failed deployments roll back before a human notices.
• Pipeline sprawl: Pipeline templates and policy-as-code let platform teams standardize delivery across hundreds of services without managing each pipeline individually.
• Onboarding time: Teams report up to 85% reduction in pipeline onboarding time compared to building from scratch.

CI/CD Pipeline: Frequently Asked Questions

What's the difference between CI/CD and a CI/CD pipeline?

CI/CD refers to the practices of Continuous Integration and Continuous Delivery or Deployment. A CI/CD pipeline is the implementation — the specific sequence of automated steps that builds, tests, and deploys your code. You can't have working CI/CD without a pipeline.

How long should a CI/CD pipeline take to run?

DORA benchmarks: elite teams achieve lead time under one hour from commit to production. If your pipeline is taking longer, look at test execution time first (usually the biggest culprit), then build cache misses, then manual approval gates that could be automated or parallelized.

What's the difference between continuous delivery and continuous deployment?

Continuous delivery keeps code in a deployable state with a manual approval gate before production. Continuous deployment removes that gate entirely. Every change that passes the pipeline ships automatically. Most teams start with continuous delivery and move toward continuous deployment as pipeline reliability and deployment verification mature.

What is GitOps and how does it relate to CI/CD?

GitOps uses Git as the source of truth for deployed state. A reconciler watches Git and applies changes to the cluster automatically. CI/CD pipelines handle everything before that: builds, tests, scans, approvals, and writing the new state to Git. They complement each other.

How can AI improve CI/CD pipelines?

In concrete terms: generating pipeline YAML from a natural language description, selecting only tests relevant to the changed code, automatically detecting deployment regressions by comparing real-time metrics to baseline, triggering rollbacks without human intervention, and surfacing root cause explanations when a build or deployment fails.

How do I secure my CI/CD pipeline?

Run SAST on every PR. Scan container images before every environment promotion. Run dependency scanning (SCA) on every build. Reserve DAST for nightly builds — it's too slow for every commit. Enforce SLSA compliance to generate provenance attestations and verify artifact integrity. Store secrets in a secrets manager, not in pipeline YAML. Run every build in an isolated, ephemeral environment.

What metrics should I track for CI/CD performance?

Start with the four DORA metrics: deployment frequency, lead time for changes, change failure rate, and MTTR. Then add pipeline-specific metrics: build duration, test pass rate, and the percentage of failed deployments caught by automated verification before a human noticed.