Harness Blog

Harness Database DevOps is introducing an open source native MongoDB executor for Liquibase Community Edition. The goal is simple: make MongoDB workflows easier, fully open, and accessible for teams already relying on Liquibase without forcing them into paid add-ons.

This launch focuses on removing friction for open source users, improving MongoDB success rates, and contributing meaningful functionality back to the community.

Why Does Liquibase MongoDB Support Matter for Open Source Users?

Teams using MongoDB often already maintain scripts, migrations, and operational workflows. However, running them reliably through Liquibase Community Edition has historically required workarounds, limited integrations, or commercial extensions.

This native executor changes that. It allows teams to:

• Run existing MongoDB scripts directly through Liquibase Community Edition.
• Avoid rewriting database workflows just to fit tooling limitations.
• Keep migrations versioned and automated alongside application CI/CD.
• Stay within a fully open source ecosystem.

This is important because MongoDB adoption continues to grow across developer platforms, fintech, eCommerce, and internal tooling. Teams want consistency: application code, infrastructure, and databases should all move through the same automation pipelines. The executor helps bring MongoDB into that standardised DevOps model.

It also reflects a broader philosophy: core database capabilities should not sit behind paywalls when the community depends on them. By open-sourcing the executor, Harness is enabling developers to move faster while keeping the ecosystem transparent and collaborative.

Liquibase MongoDB Native Executor: What It Enables In Community Edition

With the native MongoDB executor:

• Liquibase Community can execute MongoDB scripts natively
• Teams can reuse existing operational scripts
• Database changes become traceable and repeatable
• Migration workflows align with CI/CD practices

This improves the success rate for MongoDB users adopting Liquibase because the workflow becomes familiar rather than forced. Instead of adapting MongoDB to fit the tool, the tool now works with MongoDB.

How To Install The Liquibase MongoDB Extension (Step-By-Step)

1. Getting started is straightforward. The Liquibase MongoDB extension is hosted on HAR registry, which can be downloaded by using below command:

curl -L \
  "https://us-maven.pkg.dev/gar-prod-setup/harness-maven-public/io/harness/liquibase-mongodb-dbops-extension/1.1.0-4.24.0/liquibase-mongodb-dbops-extension-1.1.0-4.24.0.jar" \
  -o liquibase-mongodb-dbops-extension-1.1.0-4.24.0.jar

‍

2. Add the extension to Liquibase: Place the downloaded JAR file into the Liquibase library directory, example path: "LIQUIBASE_HOME/lib/".

3. Configure Liquibase: Update the Liquibase configuration to point to the MongoDB connection and changelog files.

4. Run migrations: Use the "liquibase update" command and Liquibase Community will now execute MongoDB scripts using the native executor.

Generating MongoDB Changelogs From A Running Database

Migration adoption often stalls when teams lack a clean way to generate changelogs from an existing database. To address this, Harness is also sharing a Python utility that mirrors the behavior of "generate-changelog" for MongoDB environments.

The script:

• Connects to a live MongoDB instance
• Reads configuration and structure
• Produces a Liquibase-compatible changelog
• Helps teams transition from unmanaged MongoDB to versioned workflows

This reduces onboarding friction significantly. Instead of starting from scratch, teams can bootstrap changelogs directly from production-like environments.  It bridges the gap between legacy MongoDB setups and modern database DevOps practices.

Why Is Harness Contributing This To Open Source?

The intent is not just to release a tool. The intent is to strengthen the open ecosystem.

Harness believes:

• Foundational database capabilities should remain accessible.
• Community users deserve production-ready tooling.
• Open contributions drive innovation faster than closed ecosystems.

By contributing a native MongoDB executor:

• Liquibase Community users gain real functionality.
• MongoDB adoption inside DevOps workflows becomes easier.
• The ecosystem remains open and collaborative.
• Higher success rate for MongoDB users adopting Liquibase Community.

This effort also reinforces Harness as an active open source contributor focused on solving real developer problems rather than monetizing basic functionality.

The Most Complete Open Source Liquibase MongoDB Integration Available Today

The native executor, combined with changelog generation support, provides:

• Script execution
• Migration automation
• Changelog creation from running environments
• CI/CD alignment

Together, these create one of the most functional open source MongoDB integrations available for Liquibase Community users. The objective is clear: make it the default path developers discover when searching for Liquibase MongoDB workflows.

Start Using and Contributing to Liquibase MongoDB Today

Discover the open-source MongoDB native executor. Teams can adopt it in their workflows, extend its capabilities, and contribute enhancements back to the project. Progress in database DevOps accelerates when the community collaborates and builds in the open.

Your developers complain about 20-minute builds while your cloud bill spirals out of control. Pipeline sprawl across teams creates security gaps you can't even see. These aren't separate problems. They're symptoms of a lack of actionable data on what actually drives velocity and cost. 

The right CI metrics transform reactive firefighting into proactive optimization. With analytics data from Harness CI, platform engineering leaders can cut build times, control spend, and maintain governance without slowing teams down.

Why Do CI Metrics Matter for Platform Engineering Leaders?

Platform teams who track the right CI metrics can quantify exactly how much developer time they're saving, control cloud spending, and maintain security standards while preserving development velocity. The importance of tracking CI/CD metrics lies in connecting pipeline performance directly to measurable business outcomes.

Reclaim Hours Through Speed Metrics

Build time, queue time, and failure rates directly translate to developer hours saved or lost. Research shows that 78% of developers feel more productive with CI, and most want builds under 10 minutes. Tracking median build duration and 95th percentile outliers can reveal your productivity bottlenecks.

Harness CI delivers builds up to 8X faster than traditional tools, turning this insight into action.

Turn Compute Minutes Into Budget Predictability

Cost per build and compute minutes by pipeline eliminate the guesswork from cloud spending. AWS CodePipeline charges $0.002 per action-execution-minute, making monthly costs straightforward to calculate from your pipeline metrics.

Measuring across teams helps you spot expensive pipelines, optimize resource usage, and justify infrastructure investments with concrete ROI.

Measure Artifact Integrity at Scale

SBOM completeness, artifact integrity, and policy pass rates ensure your software supply chain meets security standards without creating development bottlenecks. NIST and related EO 14028 guidance emphasize on machine-readable SBOMs and automated hash verification for all artifacts. 

However, measurement consistency remains challenging. A recent systematic review found that SBOM tooling variance creates significant detection gaps, with tools reporting between 43,553 and 309,022 vulnerabilities across the same 1,151 SBOMs. 

Standardized metrics help you monitor SBOM generation rates and policy enforcement without manual oversight.

10 CI/CD Metrics That Move the Needle

Not all metrics deserve your attention. Platform engineering leaders managing 200+ developers need measurements that reveal where time, money, and reliability break down, and where to fix them first.

• Performance metrics show where developers wait instead of code. High-performing organizations achieve up to 440 times faster lead times and deploy 46 times more frequently by tracking the right speed indicators.
• Cost and resource indicators expose hidden optimization opportunities. Organizations using intelligent caching can reduce infrastructure costs by up to 76% while maintaining speed, turning pipeline data into budget predictability.
• Quality and governance metrics scale security without slowing delivery. With developers increasingly handling DevOps responsibilities, compliance and reliability measurements keep distributed teams moving fast without sacrificing standards.

So what does this look like in practice? Let's examine the specific metrics.

1. Build Duration (p50/p95): Pinpointing Bottlenecks and Outliers

Build duration becomes most valuable when you track both median (p50) and 95th percentile (p95) times rather than simple averages. Research shows that timeout builds have a median duration of 19.7 minutes compared to 3.4 minutes for normal builds. That’s over five times longer. 

While p50 reveals your typical developer experience, p95 exposes the worst-case delays that reduce productivity and impact developer flow. These outliers often signal deeper issues like resource constraints, flaky tests, or inefficient build steps that averages would mask. Tracking trends in both percentiles over time helps you catch regressions before they become widespread problems. Build analytics platforms can surface when your p50 increases gradually or when p95 spikes indicate new bottlenecks. 

Keep builds under seven minutes to maintain developer engagement. Anything over 15 minutes triggers costly context switching. By monitoring both typical and tail performance, you optimize for consistent, fast feedback loops that keep developers in flow. Intelligent test selection reduces overall build durations by up to 80% by selecting and running only tests affected by the code changes, rather than running all tests.

An example of build durations dashboard (on Harness)

2. Queue Time: Measuring Infrastructure Constraints

Queue time measures how long builds wait before execution begins. This is a direct indicator of insufficient build capacity. When developers push code, builds shouldn't sit idle while runners or compute resources are tied up. Research shows that heterogeneous infrastructure with mixed processing speeds creates excessive queue times, especially when job routing doesn't account for worker capabilities. Queue time reveals when your infrastructure can't handle developer demand.

Rising queue times signal it's time to scale infrastructure or optimize resource allocation. Per-job waiting time thresholds directly impact throughput and quality outcomes. Platform teams can reduce queue time by moving to Harness Cloud's isolated build machines, implementing intelligent caching, or adding parallel execution capacity. Analytics dashboards track queue time trends across repositories and teams, enabling data-driven infrastructure decisions that keep developers productive.

3. Build Success Rate: Ensuring Pipeline Reliability

Build success rate measures the percentage of builds that complete successfully over time, revealing pipeline health and developer confidence levels. When teams consistently see success rates above 90% on their default branches, they trust their CI system to provide reliable feedback. Frequent failures signal deeper issues — flaky tests that pass and fail randomly, unstable build environments, or misconfigured pipeline steps that break under specific conditions.

Tracking success rate trends by branch, team, or service reveals where to focus improvement efforts. Slicing metrics by repository and pipeline helps you identify whether failures cluster around specific teams using legacy test frameworks or services with complex dependencies. This granular view separates legitimate experimental failures on feature branches from stability problems that undermine developer productivity and delivery confidence.

An example of Build Success/Failure Rate Dashboard (on Harness)

4. Mean Time to Recovery (MTTR): Speeding Up Incident Response

Mean time to recovery measures how fast your team recovers from failed builds and broken pipelines, directly impacting developer productivity. Research shows organizations with mature CI/CD implementations see MTTR improvements of over 50% through automated detection and rollback mechanisms. When builds fail, developers experience context switching costs, feature delivery slows, and team velocity drops. The best-performing teams recover from incidents in under one hour, while others struggle with multi-hour outages that cascade across multiple teams.

Automated alerts and root cause analysis tools slash recovery time by eliminating manual troubleshooting, reducing MTTR from 20 minutes to under 3 minutes for common failures. Harness CI's AI-powered troubleshooting surfaces failure patterns and provides instant remediation suggestions when builds break. 

5. Flaky Test Rate: Eliminating Developer Frustration

Flaky tests pass or fail non-deterministically on the same code, creating false signals that undermine developer trust in CI results. Research shows 59% of developers experience flaky tests monthly, weekly, or daily, while 47% of restarted failing builds eventually passed. This creates a cycle where developers waste time investigating false failures, rerunning builds, and questioning legitimate test results.

Tracking flaky test rate helps teams identify which tests exhibit unstable pass/fail behavior, enabling targeted stabilization efforts. Harness CI automatically detects problematic tests through failure rate analysis, quarantines flaky tests to prevent false alarms, and provides visibility into which tests exhibit the highest failure rates. This reduces developer context switching and restores confidence in CI feedback loops.

6. Cost Per Build: Controlling CI Infrastructure Spend

Cost per build divides your monthly CI infrastructure spend by the number of successful builds, revealing the true economic impact of your development velocity. CI/CD pipelines consume 15-40% of overall cloud infrastructure budgets, with per-run compute costs ranging from $0.40 to $4.20 depending on application complexity, instance type, region, and duration. This normalized metric helps platform teams compare costs across different services, identify expensive outliers, and justify infrastructure investments with concrete dollar amounts rather than abstract performance gains.

Automated caching and ephemeral infrastructure deliver the biggest cost reductions per build. Intelligent caching automatically stores dependencies and Docker layers. This cuts repeated download and compilation time that drives up compute costs.

Ephemeral build machines eliminate idle resource waste. They spin up fresh instances only when the queue builds, then terminate immediately after completion. Combine these approaches with right-sized compute types to reduce infrastructure costs by 32-43% compared to oversized instances.

7. Cache Hit Rate: Accelerating Builds With Smart Caching

Cache hit rate measures what percentage of build tasks can reuse previously cached results instead of rebuilding from scratch. When teams achieve high cache hit rates, they see dramatic build time reductions. Docker builds can drop from five to seven minutes to under 90 seconds with effective layer caching. Smart caching of dependencies like node_modules, Docker layers, and build artifacts creates these improvements by avoiding expensive regeneration of unchanged components.

Harness Build and Cache Intelligence eliminates the manual configuration overhead that traditionally plagues cache management. It handles dependency caching and Docker layer reuse automatically. No complex cache keys or storage management required.

Measure cache effectiveness by comparing clean builds against fully cached runs. Track hit rates over time to justify infrastructure investments and detect performance regressions.

8. Test Cycle Time: Optimizing Feedback Loops

Test cycle time measures how long it takes to run your complete test suite from start to finish. This directly impacts developer productivity because longer test cycles mean developers wait longer for feedback on their code changes. When test cycles stretch beyond 10-15 minutes, developers often switch context to other tasks, losing focus and momentum. Recent research shows that optimized test selection can accelerate pipelines by 5.6x while maintaining high failure detection rates.

Smart test selection optimizes these feedback loops by running only tests relevant to code changes. Harness CI Test Intelligence can slash test cycle time by up to 80% using AI to identify which tests actually need to run. This eliminates the waste of running thousands of irrelevant tests while preserving confidence in your CI deployments.

9. Pipeline Failure Cause Distribution: Prioritizing Remediation

Categorizing pipeline issues into domains like code problems, infrastructure incidents, and dependency conflicts transforms chaotic build logs into actionable insights. Harness CI's AI-powered troubleshooting provides root cause analysis and remediation suggestions for build failures. This helps platform engineers focus remediation efforts on root causes that impact the most builds rather than chasing one-off incidents.

Visualizing issue distribution reveals whether problems are systemic or isolated events. Organizations using aggregated monitoring can distinguish between infrastructure spikes and persistent issues like flaky tests. Harness CI's analytics surface which pipelines and repositories have the highest failure rates. Platform teams can reduce overall pipeline issues by 20-30%.

10. Artifact Integrity Coverage: Securing the Software Supply Chain

Artifact integrity coverage measures the percentage of builds that produce signed, traceable artifacts with complete provenance documentation. This tracks whether each build generates Software Bills of Materials (SBOMs), digital signatures, and documentation proving where artifacts came from. While most organizations sign final software products, fewer than 20% deliver provenance data and only 3% consume SBOMs for dependency management. This makes the metric a leading indicator of supply chain security maturity.

Harness CI automatically generates SBOMs and attestations for every build, ensuring 100% coverage without developer intervention. The platform's SLSA L3 compliance capabilities generate verifiable provenance and sign artifacts using industry-standard frameworks. This eliminates the manual processes and key management challenges that prevent consistent artifact signing across CI pipelines.

Steps to Track CI/CD Metrics and Turn Insights Into Action

Tracking CI metrics effectively requires moving from raw data to measurable improvements. The most successful platform engineering teams build a systematic approach that transforms metrics into velocity gains, cost reductions, and reliable pipelines.

Step 1: Standardize Pipeline Metadata Across Teams

Tag every pipeline with service name, team identifier, repository, and cost center. This standardization creates the foundation for reliable aggregation across your entire CI infrastructure. Without consistent tags, you can't identify which teams drive the highest costs or longest build times.

Implement naming conventions that support automated analysis. Use structured formats like team-service-environment for pipeline names and standardize branch naming patterns. Centralize this metadata using automated tag enforcement to ensure organization-wide visibility.

Step 2: Automate Metric Collection and Visualization

Modern CI platforms eliminate manual metric tracking overhead. Harness CI provides dashboards that automatically surface build success rates, duration trends, and failure patterns in real-time. Teams can also integrate with monitoring stacks like Prometheus and Grafana for live visualization across multiple tools.

Configure threshold-based alerts for build duration spikes or failure rate increases. This shifts you from fixing issues after they happen to preventing them entirely.

Step 3: Analyze Metrics and Identify Optimization Opportunities

Focus on p95 and p99 percentiles rather than averages to identify critical performance outliers. Drill into failure causes and flaky tests to prioritize fixes with maximum developer impact. Categorize pipeline failures by root cause — environment issues, dependency problems, or test instability — then target the most frequent culprits first.

Benchmark cost per build and cache hit rates to uncover infrastructure savings. Optimized caching and build intelligence can reduce build times by 30-40% while cutting cloud expenses.

Step 4: Operationalize Improvements With Governance and Automation

Standardize CI pipelines using centralized templates and policy enforcement to eliminate pipeline sprawl. Store reusable templates in a central repository and require teams to extend from approved templates. This reduces maintenance overhead while ensuring consistent security scanning and artifact signing.

Establish Service Level Objectives (SLOs) for your most impactful metrics: build duration, queue time, and success rate. Set measurable targets like "95% of builds complete within 10 minutes" to drive accountability. Automate remediation wherever possible — auto-retry for transient failures, automated cache invalidation, and intelligent test selection to skip irrelevant tests.

Make Your CI Metrics Work

The difference between successful platform teams and those drowning in dashboards comes down to focus. Elite performers track build duration, queue time, flaky test rates, and cost per build because these metrics directly impact developer productivity and infrastructure spend.

Start with the measurements covered in this guide, establish baselines, and implement governance that prevents pipeline sprawl. Focus on the metrics that reveal bottlenecks, control costs, and maintain reliability — then use that data to optimize continuously.

Ready to transform your CI metrics from vanity to velocity? Experience how Harness CI accelerates builds while cutting infrastructure costs.

Continuous Integration Metrics FAQ

Platform engineering leaders often struggle with knowing which metrics actually move the needle versus creating metric overload. These answers focus on metrics that drive measurable improvements in developer velocity, cost control, and pipeline reliability.

What separates actionable CI metrics from vanity metrics?

Actionable metrics directly connect to developer experience and business outcomes. Build duration affects daily workflow, while deployment frequency impacts feature delivery speed. Vanity metrics look impressive, but don't guide decisions. Focus on measurements that help teams optimize specific bottlenecks rather than general health scores.

Which CI metrics have the biggest impact on developer productivity?

Build duration, queue time, and flaky test rate directly affect how fast developers get feedback. While coverage monitoring dominates current practices, build health and time-to-fix-broken-builds offer the highest productivity gains. Focus on metrics that reduce context switching and waiting.

How do CI metrics help reduce infrastructure costs without sacrificing quality?

Cost per build and cache hit rate reveal optimization opportunities that maintain quality while cutting spend. Intelligent caching and optimized test selection can significantly reduce both build times and infrastructure costs. Running only relevant tests instead of entire suites cuts waste without compromising coverage.

What's the most effective way to start tracking CI metrics across different tools?

Begin with pipeline metadata standardization using consistent tags for service, team, and cost center. Most CI platforms provide basic metrics through built-in dashboards. Start with DORA metrics, then add build-specific measurements as your monitoring matures.

How often should teams review CI metrics and take action?

Daily monitoring of build success rates and queue times enables immediate issue response. Weekly reviews of build duration trends and monthly cost analysis drive strategic improvements. Automated alerts for threshold breaches prevent small problems from becoming productivity killers.

Modern unit testing in CI/CD can help teams avoid slow builds by using smart strategies. Choosing the right tests, running them in parallel, and using intelligent caching all help teams get faster feedback while keeping code quality high.

Platforms like Harness CI use AI-powered test intelligence to reduce test cycles by up to 80%, showing what’s possible with the right tools. This guide shares practical ways to speed up builds and improve code quality, from basic ideas to advanced techniques that also lower costs.

What Is a Unit Test?

Knowing what counts as a unit test is key to building software delivery pipelines that work.

The Smallest Testable Component

A unit test looks at a single part of your code, such as a function, class method, or a small group of related components. The main point is to test one behavior at a time. Unit tests are different from integration tests because they look at the logic of your code. This makes it easier to figure out what went wrong if something goes wrong.

Isolation Drives Speed and Reliability

Unit tests should only check code that you wrote and not things like databases, file systems, or network calls. This separation makes tests quick and dependable. Tests that don't rely on outside services run in milliseconds and give the same results no matter where they are run, like on your laptop or in a CI pipeline.

Foundation for CI/CD Quality Gates

Unit tests are one of the most important part of continuous integration in CI/CD pipelines because they show problems right away after code changes. Because they are so fast, developers can run them many times a minute while they are coding. This makes feedback loops very tight, which makes it easier to find bugs and stops them from getting to later stages of the pipeline.

Unit Testing Strategies: Designing for Speed and Reliability

Teams that run full test suites on every commit catch problems early by focusing on three things: making tests fast, choosing the right tests, and keeping tests organized. Good unit testing helps developers stay productive and keeps builds running quickly.

Deterministic Tests for Every Commit

Unit tests should finish in seconds, not minutes, so that they can be quickly checked. Google's engineering practices say that tests need to be "fast and reliable to give engineers immediate feedback on whether a change has broken expected behavior." To keep tests from being affected by outside factors, use mocks, stubs, and in-memory databases. Keep commit builds to less than ten minutes, and unit tests should be the basis of this quick feedback loop.

Intelligent Test Selection

As projects get bigger, running all tests on every commit can slow teams down. Test Impact Analysis looks at coverage data to figure out which tests really check the code that has been changed. AI-powered test selection chooses the right tests for you, so you don't have to guess or sort them by hand.

Parallelization and Caching

To get the most out of your infrastructure, use selective execution and run tests at the same time. Divide test suites into equal-sized groups and run them on different machines simultaneously. Smart caching of dependencies, build files, and test results helps you avoid doing the same work over and over. When used together, these methods cut down on build time a lot while keeping coverage high.

Standardized Organization for Scale

Using consistent names, tags, and organization for tests helps teams track performance and keep quality high as they grow. Set clear rules for test types (like unit, integration, or smoke) and use names that show what each test checks. Analytics dashboards can spot flaky tests, slow tests, and common failures. This helps teams improve test suites and keep things running smoothly without slowing down developers.

Unit Test Example: From Code to Assertion

A good unit test uses the Arrange-Act-Assert pattern. For example, you might test a function that calculates order totals with discounts:

def test_apply_discount_to_order_total():
   # Arrange: Set up test data
   order = Order(items=[Item(price=100), Item(price=50)])
   discount = PercentageDiscount(10)
   
   # Act: Execute the function under test
   final_total = order.apply_discount(discount)
   
   # Assert: Verify expected outcome
   assert final_total == 135  # 150 - 10% discount

In the Arrange phase, you set up the objects and data you need. In the Act phase, you call the method you want to test. In the Assert phase, you check if the result is what you expected.

Testing Edge Cases

Real-world code needs to handle more than just the usual cases. Your tests should also check edge cases and errors:

def test_apply_discount_with_empty_cart_returns_zero():
   order = Order(items=[])
   discount = PercentageDiscount(10)
   
   assert order.apply_discount(discount) == 0
​
def test_apply_discount_rejects_negative_percentage():
   order = Order(items=[Item(price=100)])
   
   with pytest.raises(ValueError):
       PercentageDiscount(-5)

Notice the naming style: test_apply_discount_rejects_negative_percentage clearly shows what’s being tested and what should happen. If this test fails in your CI pipeline, you’ll know right away what went wrong, without searching through logs.

Benefits of Unit Testing: Building Confidence and Saving Time

When teams want faster builds and fewer late-stage bugs, the benefits of unit testing are clear. Good unit tests help speed up development and keep quality high.

• Catch regressions right away: Unit tests run in seconds and find breaking changes before they get to integration or production environments.
• Allow fearless refactoring: A strong set of tests gives you the confidence to change code without adding bugs you didn't expect.
• Cut down on costly debugging: Research shows that unit tests cover a lot of ground and find bugs early when fixing them is cheapest.
• Encourage modular design: Writing code that can be tested naturally leads to better separation of concerns and a cleaner architecture.

When you use smart test execution in modern CI/CD pipelines, these benefits get even bigger.

Disadvantages of Unit Testing: Recognizing the Trade-Offs

Unit testing is valuable, but knowing its limits helps teams choose the right testing strategies. These downsides matter most when you’re trying to make CI/CD pipelines faster and more cost-effective.

• Maintenance overhead grows as automated tests expand, requiring ongoing effort to update brittle or overly granular tests.
• False confidence occurs when high unit test coverage hides integration problems and system-level failures.
• Slow execution times can bottleneck CI pipelines when test collections take hours instead of minutes to complete.
• Resource allocation shifts developer time from feature work to test maintenance and debugging flaky tests.
• Coverage gaps appear in areas like GUI components, external dependencies, and complex state interactions.

Research shows that automatically generated tests can be harder to understand and maintain. Studies also show that statement coverage doesn’t always mean better bug detection.

Industry surveys show that many organizations have trouble with slow test execution and unclear ROI for unit testing. Smart teams solve these problems by choosing the right tests, using smart caching, and working with modern CI platforms that make testing faster and more reliable.

How Do Developers Use Unit Tests in Real Workflows?

Developers use unit tests in three main ways that affect build speed and code quality. These practices turn testing into a tool that catches problems early and saves time on debugging.

Test-Driven Development and Rapid Feedback Loops

Before they start coding, developers write unit tests. They use test-driven development (TDD) to make the design better and cut down on debugging. According to research, TDD finds 84% of new bugs, while traditional testing only finds 62%. This method gives you feedback right away, so failing tests help you decide what to do next.

Regression Prevention and Bug Validation

Unit tests are like automated guards that catch bugs when code changes. Developers write tests to recreate bugs that have been reported, and then they check that the fixes work by running the tests again after the fixes have been made. Automated tools now generate test cases from issue reports. They are 30.4% successful at making tests that fail for the exact problem that was reported. To stop bugs that have already been fixed from coming back, teams run these regression tests in CI pipelines.

Strategic Focus on Business Logic and Public APIs

Good developer testing doesn't look at infrastructure or glue code; it looks at business logic, edge cases, and public interfaces. Testing public methods and properties is best; private details that change often should be left out. Test doubles help developers keep business logic separate from systems outside of their control, which makes tests more reliable. Integration and system tests are better for checking how parts work together, especially when it comes to things like database connections and full workflows.

Unit Testing Best Practices: Maximizing Value, Minimizing Pain

Slow, unreliable tests can slow down CI and hurt productivity, while also raising costs. The following proven strategies help teams check code quickly and cut both build times and cloud expenses.

• Write fast, isolated tests that run in milliseconds and avoid external dependencies like databases or APIs.
• Use descriptive test names that clearly explain the behavior being tested, not implementation details.
• Run only relevant tests using selective execution to cut cycle times by up to 80%.
• Monitor test health with failure analytics to identify flaky or slow tests before they impact productivity.
• Refactor tests regularly alongside production code to prevent technical debt and maintain suite reliability.

Types of Unit Testing: Manual vs. Automated

Choosing between manual and automated unit testing directly affects how fast and reliable your pipeline is.

Manual Unit Testing: Flexibility with Limitations

Manual unit testing means developers write and run tests by hand, usually early in development or when checking tricky edge cases that need human judgment. This works for old systems where automation is hard or when you need to understand complex behavior. But manual testing can’t be repeated easily and doesn’t scale well as projects grow.

Automated Unit Testing: Speed and Consistency at Scale

Automated testing transforms test execution into fast, repeatable processes that integrate seamlessly with modern development workflows. Modern platforms leverage AI-powered optimization to run only relevant tests, cutting cycle times significantly while maintaining comprehensive coverage.

Why High-Velocity Teams Prioritize Automation

Fast-moving teams use automated unit testing to keep up speed and quality. Manual testing is still useful for exploring and handling complex cases, but automation handles the repetitive checks that make deployments reliable and regular.

Difference Between Unit Testing and Other Types of Testing

Knowing the difference between unit, integration, and other test types helps teams build faster and more reliable CI/CD pipelines. Each type has its own purpose and trade-offs in speed, cost, and confidence.

Unit Tests: Fast and Isolated Validation

Unit tests are the most important part of your testing plan. They test single functions, methods, or classes without using any outside systems. You can run thousands of unit tests in just a few minutes on a good machine. This keeps you from having problems with databases or networks and gives you the quickest feedback in your pipeline.

Integration Tests: Validating Component Interactions

Integration testing makes sure that the different parts of your system work together. There are two main types of tests: narrow tests that use test doubles to check specific interactions (like testing an API client with a mock service) and broad tests that use real services (like checking your payment flow with real payment processors). Integration tests use real infrastructure to find problems that unit tests might miss.

End-to-End Tests: Complete User Journey Validation

The top of the testing pyramid is end-to-end tests. They mimic the full range of user tasks in your app. These tests are the most reliable, but they take a long time to run and are hard to fix. Unit tests can find bugs quickly, but end-to-end tests may take days to find the same bug. This method works, but it can be brittle.

The Test Pyramid: Balancing Speed and Coverage

The best testing strategy uses a pyramid: many small, fast unit tests at the bottom, some integration tests in the middle, and just a few end-to-end tests at the top.

Workflow of Unit Testing in CI/CD Pipelines

Modern development teams use a unit testing workflow that balances speed and quality. Knowing this process helps teams spot slow spots and find ways to speed up builds while keeping code reliable.

The Standard Development Cycle

Before making changes, developers write code on their own computers and run unit tests. They run tests on their own computers to find bugs early, and then they push the code to version control so that CI pipelines can take over. This step-by-step process helps developers stay productive by finding problems early, when they are easiest to fix.

Automated CI Pipeline Execution

Once code is in the pipeline, automation tools run unit tests on every commit and give feedback right away. If a test fails, the pipeline stops deployment and lets developers know right away. This automation stops bad code from getting into production. Research shows this method can cut critical defects by 40% and speed up deployments.

Accelerating the Workflow

Modern CI platforms use Test Intelligence to only run the tests that are affected by code changes in order to speed up this process. Parallel testing runs test groups in different environments at the same time. Smart caching saves dependencies and build files so you don't have to do the same work over and over. These steps can help keep coverage high while lowering the cost of infrastructure.

Results Analysis and Continuous Improvement

Teams analyze test results through dashboards that track failure rates, execution times, and coverage trends. Analytics platforms surface patterns like flaky tests or slow-running suites that need attention. This data drives decisions about test prioritization, infrastructure scaling, and process improvements. Regular analysis ensures the unit testing approach continues to deliver value as codebases grow and evolve.

Unit Testing Techniques: Tools for Reliable, Maintainable Tests

Using the right unit testing techniques can turn unreliable tests into a reliable way to speed up development. These proven methods help teams trust their code and keep CI pipelines running smoothly:

• Replace slow external dependencies with controllable test doubles that run consistently.
• Generate hundreds of test cases automatically to find edge cases you'd never write manually.
• Run identical test logic against multiple inputs to expand coverage without extra maintenance.
• Capture complex output snapshots to catch unintended changes in data structures.
• Verify behavior through isolated components that focus tests on your actual business logic.

These methods work together to build test suites that catch real bugs and stay easy to maintain as your codebase grows.

Isolation Through Test Doubles

As we've talked about with CI/CD workflows, the first step to good unit testing is to separate things. This means you should test your code without using outside systems that might be slow or not work at all. Dependency injection is helpful because it lets you use test doubles instead of real dependencies when you run tests.

It is easier for developers to choose the right test double if they know the differences between them. Fakes are simple working versions, such as in-memory databases. Stubs return set data that can be used to test queries. Mocks keep track of what happens so you can see if commands work as they should.

This method makes sure that tests are always quick and accurate, no matter when you run them. Tests run 60% faster and there are a lot fewer flaky failures that slow down development when teams use good isolation.

Teams need more ways to get more test coverage without having to do more work, in addition to isolation. You can set rules that should always be true with property-based testing, and it will automatically make hundreds of test cases. This method is great for finding edge cases and limits that manual tests might not catch.

Expanding Coverage with Smart Generation

Parameterized testing gives you similar benefits, but you have more control over the inputs. You don't have to write extra code to run the same test with different data. Tools like xUnit's Theory and InlineData make this possible. This helps find more bugs and makes it easier to keep track of your test suite.

Both methods work best when you choose the right tests to run. You only run the tests you need, so platforms that know which tests matter for each code change give you full coverage without slowing things down.

Verifying Complex Outputs

The last step is to test complicated data, such as JSON responses or code that was made. Golden tests and snapshot testing make things easier by saving the expected output as reference files, so you don't have to do complicated checks.

If your code’s output changes, the test fails and shows what’s different. This makes it easy to spot mistakes, and you can approve real changes by updating the snapshot. This method works well for testing APIs, config generators, or any code that creates structured output.

Teams that use full automated testing frameworks see code coverage go up by 32.8% and catch 74.2% more bugs per build. Golden tests help by making it easier to check complex cases that would otherwise need manual testing.

The main thing is to balance thoroughness with easy maintenance. Golden tests should check real behavior, not details that change often. When you get this balance right, you’ll spend less time fixing bugs and more time building features.

Unit Testing Tools: Frameworks That Power Modern Teams

Picking the right unit testing tools helps your team write tests efficiently, instead of wasting time on flaky tests or slow builds. The best frameworks work well with your language and fit smoothly into your CI/CD process.

• JUnit and TestNG dominate Java environments, with TestNG offering advanced features like parallel execution and seamless pipeline integration.
• pytest leads Python testing environments with powerful fixtures and minimal boilerplate, making it ideal for teams prioritizing developer experience.
• Jest provides zero-configuration testing for JavaScript/TypeScript projects, with built-in mocking and snapshot capabilities.
• RSpec delivers behavior-driven development for Ruby teams, emphasizing readable test specifications.

Modern teams use these frameworks along with CI platforms that offer analytics and automation. This mix of good tools and smart processes turns testing from a bottleneck into a productivity boost.

Transform Your Development Velocity Today

Smart unit testing can turn CI/CD from a bottleneck into an advantage. When tests are fast and reliable, developers spend less time waiting and more time releasing code. Harness Continuous Integration uses Test Intelligence, automated caching, and isolated build environments to speed up feedback without losing quality.

Want to speed up your team? Explore Harness CI and see what's possible.

Filtering data is at the heart of developer productivity. Whether you’re looking for failed builds, debugging a service or analysing deployment patterns, the ability to quickly slice and dice execution data is critical.

At Harness, users across CI, CD and other modules rely on filtering to navigate complex execution data by status, time range, triggers, services and much more. While our legacy filtering worked, it had major pain points — hidden drawers, inconsistent behaviour and lost state on refresh — that slowed both developers and users.

This blog dives into how we built a new Filters component system in React: a reusable, type-safe and feature-rich framework that powers the filtering experience on the Execution Listing page (and beyond).

Prefer Watching? Here’s the Talk

The Starting Point: Challenges with Our Legacy Filters

Our old implementation revealed several weaknesses as Harness scaled:

• Poor Discoverability and UX: Filters were hidden in a side panel, disrupting workflow and making applied filters non-glanceable. Users didn’t get feedback until the filter was applied/saved.
• Inconsistency Across Modules: Custom logic in modules like CI and CD led to confusing behavioural differences.
• High Developer Overhead: Adding new filters was cumbersome, requiring edits to multiple files with brittle boilerplate.

These problems shaped our success criteria: discoverability, smooth UX, consistent behaviour, reusable design and decoupled components.

The Evolution of Filters: A Design Journey

Building a truly reusable and powerful filtering system required exploration and iteration. Our journey involved several key stages and learning from the pitfalls of each:

Iteration 1: React Components (Conditional Rendering)

Shifted to React functional components but kept logic centralised in the FilterFramework. Each filter was conditionally rendered based on visibleFilters array. Framework fetched filter options and passed them down as props.

COMPONENT FilterFramework:
    STATE activeFilters = {}
    STATE visibleFilters = []
    STATE filterOptions = {}
    
    ON visibleFilters CHANGE:
        FOR EACH filter IN visibleFilters:
            IF filterOptions[filter] NOT EXISTS:
                options = FETCH filterData(filter)
                filterOptions[filter] = options
    
    ON activeFilters CHANGE:
        makeAPICall(activeFilters)
    
    RENDER:
        <AllFilters setVisibleFilters={setVisibleFilters} />
        
        IF 'services' IN visibleFilters:
            <DropdownFilter 
                name="Services"
                options={filterOptions.services}
                onAdd={updateActiveFilters}
                onRemove={removeFromVisible}
            />
        
        IF 'environments' IN visibleFilters:
            <DropdownFilter ... />

Pitfalls: Adding new filters required changes in multiple places, creating a maintenance nightmare and poor developer experience. The framework had minimal control over filter implementation, lacked proper abstraction and scattered filter logic across the codebase, making it neither “stupid-proof” nor scalable.

Iteration 2: React.cloneElement Pattern

Improved the previous approach by accepting filters as children and using React.cloneElement to inject callbacks (onAdd, onRemove) from the parent framework. This gave developers a cleaner API to add filters.

children.forEach(child => {
  if (visibleFilters.includes(child.props.filterKey)) {
    return React.cloneElement(child, {
      onAdd: (label, value) => {
        activeFilters[child.props.filterKey].push({ label, value });
      },
      onRemove: () => {
        delete activeFilters[child.props.filterKey];
      }
    });
  }
});

Pitfalls: React.cloneElement is an expensive operation that causes performance issues with frequent re-renders and it’s considered an anti-pattern by the React team. The approach tightly coupled filters to the framework’s callback signature, made prop flow implicit and difficult to debug and created type safety issues since TypeScript struggles with dynamically injected props.

Final Solution: Context API

The winning design uses React Context API to provide filter state and actions to child components. Individual filters access setValue and removeFilter via useFiltersContext() hook. This decouples filters from the framework while maintaining control.

COMPONENT Filters({ children, onChange }):
    STATE filtersMap = {}           // { search: { value, query, state } }
    STATE filtersOrder = []         // ['search', 'status']

    FUNCTION updateFilter(key, newValue):
        serialized = parser.serialize(newValue)   // Type → String
        filtersMap[key] = { value: newValue, query: serialized }
        updateURL(serialized)
        onChange(allValues)

    ON URL_CHANGE:
        parsed = parser.parse(urlString)          // String → Type
        filtersMap[key] = { value: parsed, query: urlString }

    RENDER:
        <Context.Provider value={{ updateFilter, filtersMap }}>
            {children}
        </Context.Provider>
END COMPONENT

Benefits: This solution eliminated the performance overhead of cloneElement, decoupled filters from framework internals and made it easy to add new filters without touching framework code. The Context API provides clear data flow that’s easy to debug and test, with type safety through TypeScript.

Inversion of Control (IoC)

The Context API in React unlocks something truly powerful — Inversion of Control (IoC). This design principle is about delegating control to a framework instead of managing every detail yourself. It’s often summed up by the Hollywood Principle: “Don’t call us, we’ll call you.”

In React, this translates to building flexible components that let the consumer decide what to render, while the component itself handles how and when it happens.

Our Filters framework applies this principle: you don’t have to manage when to update state or synchronise the URL. You simply define your filter components and the framework orchestrates the rest — ensuring seamless, predictable updates without manual intervention.

How Filters Inverts Control

Our Filters framework demonstrates Inversion of Control in three key ways.

• Logic via Props: The framework doesn’t know how to save filters or fetch data — the parent injects those functions. The framework decides when to call them, but the parent defines what they do.
• Content via Children (Composition): The parent decides which filters to render.
• Actions via Callbacks: The framework triggers callbacks when users type, select or apply filters, but it’s your code that decides what happens next — fetch data, update cache or send analytics.

The result? A single, reusable Filters component that works across pipelines, services, deployments or repositories. By separating UI logic from business logic, we gain flexibility, testability and cleaner architecture — the true power of Inversion of Control.

COMPONENT DemoPage:
    STATE filterValues
    FilterHandler = createFilters()

    FUNCTION applyFilters(data, filters):
        result = data
        IF filters.onlyActive == true:
            result = result WHERE item.status == "Active"
        RETURN result

    filteredData = applyFilters(SAMPLE_DATA, filterValues)

    RENDER:
        <RouterContextProvider>
            <FilterHandler onChange = (updatedFilters) => SET filterValues = updatedFilters>
                
                // Dropdown to add filters dynamically
                <FilterHandler.Dropdown>
                    RENDER FilterDropdownMenu with available filters
                </FilterHandler.Dropdown>

                // Active filters section
                <FilterHandler.Content>
                    <FilterHandler.Component parser = booleanParser filterKey = "onlyActive">
                        RENDER CustomActiveOnlyFilter
                    </FilterHandler.Component>
                </FilterHandler.Content>

            </FilterHandler>

            RENDER DemoTable(filteredData)
        </RouterContextProvider>
END COMPONENT

The URL Problem

One of the key technical challenges in building a filtering system is URL synchronization. Browsers only understand strings, yet our applications deal with rich data types — dates, booleans, arrays and more. Without a structured solution, each component would need to manually convert these values, leading to repetitive, error-prone code.

The solution is our parser interface, a lightweight abstraction with just two methods: parse and serialize.

• parse converts a URL string into the type your app needs.
• serialize does the opposite, turning that typed value back into a string for the URL.

This bidirectional system runs automatically — parsing when filters load from the URL and serialising when users update filters.

const booleanParser: Parser<boolean> = {
  parse: (value: string) => value === 'true',   // "true" → true
  serialize: (value: boolean) => String(value)  // true → "true"
}

FiltersMap — The State Hub

At the heart of our framework lies the FiltersMap — a single, centralized object that holds the complete state of all active filters. It acts as the bridge between your React components and the browser, keeping UI state and URL state perfectly in sync.

Each entry in the FiltersMap contains three key fields:

• Value — the parsed, typed data your components actually use (e.g. Date objects, arrays, booleans).
• Query — the serialized string representation that’s written to the URL.
• State — the filter’s lifecycle status: hidden, visible or actively filtering.

You might ask — why store both the typed value and its string form? The answer is performance and reliability. If we only stored the URL string, every re-render would require re-parsing, which quickly becomes inefficient for complex filters like multi-selects. By storing both, we parse only once — when the value changes — and reuse the typed version afterward. This ensures type safety, faster URL synchronization and a clean separation between UI behavior and URL representation. The result is a system that’s predictable, scalable, and easy to maintain.

interface FilterType<T = any> {
  value?: T              // The actual filter value
  query?: string         // Serialized string for URL
  state: FilterStatus    // VISIBLE | FILTER_APPLIED | HIDDEN
}

The Journey of a Filter Value

Let’s trace how a filter value moves through the system — from user interaction to URL synchronization.

It all starts when a user interacts with a filter component — for example, selecting a date. This triggers an onChange event with a typed value, such as a Date object. Before updating the state, the parser’s serialize method converts that typed value into a URL-safe string.

The framework then updates the FiltersMap with both versions:

• the typed value under value and
• the serialized string under query.

From here, two things happen simultaneously:

1. The onChange callback fires, passing typed values back to the parent component — allowing the app to immediately fetch data or update visualizations.
2. The URL updates using the serialized query string, keeping the browser’s address bar in sync and making the current filter state instantly shareable or bookmarkable.

The reverse flow works just as seamlessly. When the URL changes — say, the user clicks the back button — the parser’s parse method converts the string back into a typed value, updates the FiltersMap and triggers a re-render of the UI.

All of this happens within milliseconds, enabling a smooth, bidirectional synchronization between the application state and the URL — a crucial piece of what makes the Filters framework feel so effortless.

Conclusion

For teams tackling similar challenges — complex UI state management, URL synchronization and reusable component design — this architecture offers a practical blueprint to build upon. The patterns used are not specific to Harness; they are broadly applicable to any modern frontend system that requires scalable, stateful and user-driven filtering.

The team’s core objectives — discoverability, smooth UX, consistent behavior, reusable design and decoupled elements — directly shaped every architectural decision. Through Inversion of Control, the framework manages the when and how of state updates, lifecycle events and URL synchronization, while developers define the what — business logic, API calls and filter behavior.

By treating the URL as part of the filter state, the architecture enables shareability, bookmarkability and native browser history support. The Context API serves as the control distribution layer, removing the need for prop drilling and allowing deeply nested components to seamlessly access shared logic and state.

Ultimately, Inversion of Control also paved the way for advanced capabilities such as saved filters, conditional rendering, and sticky filters — all while keeping the framework lightweight and maintainable. This approach demonstrates how clear objectives and sound architectural principles can lead to scalable, elegant solutions in complex UI systems.

For the last decade, the holy grail of DevOps has been Automation. We spent years writing Bash scripts to move files, Terraform to provision servers, and Ansible to configure them. And for a while, it felt like magic.

But any seasoned engineer knows the dirty secret of automation: it is brittle.

Automation is deterministic. It only does exactly what you tell it to do. It has no brain. It cannot reason. If an error occurs that isn't explicitly handled in your try/catch block, the script fails, the pipeline breaks, and a human gets paged at 3 AM to fix a problem that a machine should have understood.

We are now standing on the precipice of a new era. We are graduating from Generative AI, which writes the script for you, to Agentic AI, which runs the script, sees the error, analyzes the logs, debugs the root cause, and applies the fix.

As a Solution Architect, I see this not merely as a tool upgrade, but as a fundamental shift in our operational philosophy. We are transitioning from managing servers to managing Cognitive Architectures.

Beyond the Chatbot: The Anatomy of an Agent

To understand where we are going, we have to look beyond the "Chat with PDF" use cases. A true DevOps Agent isn't a browser window you type into; it is an autonomous control loop hardwired directly into your infrastructure.

Think of it as building a synthetic brain for your operations. We define this system using the C-P-A Model (Context, Planning, Action). It transforms an LLM from a text generator into a decision engine.

1. Perception (The Sensors): An agent cannot act on what it cannot see. Unlike a human reading a screen, an agent ingests high-cardinality data, logs, metrics, and traces, converting them into embeddings to detect anomalies that would slip past a regex filter.‍
2. Memory (The Context): Scripts are amnesiacs; they run and forget. Agents have memory. Using RAG (Retrieval-Augmented Generation), they pull from a Vector Database containing your entire history of runbooks, architectural diagrams, and past incident reports. They "remember" how you fixed this last time.‍
3. Reasoning (The Orchestrator): This is the core differentiator. Using patterns like ReAct (Reason + Act) or Chain-of-Thought, the agent breaks a complex alert—like "High Latency in US-East"—into a step-by-step investigation plan, hypothesizing root causes before touching a single server.‍
4. Tool Use (The Actuators): Finally, the agent needs hands. Through a secure tool interface, it executes CLI commands (Kubectl, AWS CLI, Git) to effect change, verifying the output of every command just like a human engineer would.

Architectural Diagram: The Cognitive Loop

The Architect's Force Multiplier: From YAML Herder to System Designer

For us architects, Agentic AI solves our biggest pain point: The disconnect between design and reality.

Today, we draw beautiful diagrams in Miro or Visio. Then, developers write Terraform that mostly matches the diagram. Six months later, the infrastructure has drifted, the documentation is stale, and the architecture diagram is a lie.

Agentic AI acts as the "Force Multiplier" that keeps the map aligned with the territory.

1. The "Living Architecture" Engine

Instead of manually updating diagrams, we can deploy Observer Agents. These agents continuously scan the cloud estate (via Read-Only APIs) and reverse-engineer the current state into architectural models.

• The Shift: If a developer manually spins up an RDS instance, the Agent detects the drift, updates the architectural record, and alerts the architect that the implementation has deviated from the standard.

2. Intent-Based Provisioning

We stop writing boilerplate. As an architect, I shouldn't care about the syntax of an AWS::EC2::Instance. I care about the properties of the system.

• The Workflow: I define the Intent: "I need a resilient, PCI-compliant message queue that can handle 10k TPS."
• The Agent: The agent (trained on your organization's “Golden Standards”) generates the Terraform, ensures it uses approved modules, and verifies compliance before a human ever reviews the PR.

3. Continuous Cost & Security Optimization

Humans are bad at optimization because it requires analyzing thousands of data points. Agents thrive here.

• Scenario: An "Optimizer Agent" notices a cluster is over-provisioned by 40%. It doesn't just send a report; it proposes a new Terraform plan to right-size the instance types, calculates the exact savings ($4,200/month), and opens a Pull Request for the architect to approve.

‍

Visualizing the Architect's New Workflow

The Three Horizons of Adoption

We won't wake up tomorrow with a fully autonomous SRE team. This transition will happen in waves or "Horizons", each requiring a higher degree of trust and architectural maturity.

Horizon 1: The "Augmented Operator" (Today)

We are already living here. Agents exist as "Sidecars" to the engineer.

• The Workflow: You are the driver; the AI is the navigator. You ask, "Why is this pod crashing?" The Agent queries logs, correlates a MemoryLimitExceeded error with a recent config change, and proposes a fix.‍
• The Architecture: IDE extensions and CLI wrappers. The human creates the intent and approves the action.

Horizon 2: Agent Swarms & Task Autonomy (1-2 Years)

This is where things get interesting. We start giving agents specific scopes to handle repetitive tasks autonomously. We also see the rise of Agent Swarms—specialized agents collaborating.

• The Workflow: A "Security" agent identifies a CVE. It doesn't just alert you; it creates a ticket and passes it to a "Developer" agent. That agent creates a branch, bumps the version, and passes the PR to a "QA" agent to run tests. The human steps in only at the very end to click "Merge."‍
• The Shift: We move from Human-in-the-Loop to Human-on-the-Loop. You review the output, but you don't drive the process.

Horizon 3: The "Autonomous SRE" (3-5 Years)

The game-changer. Agents act as Tier-1 support, proactively managing system health.

• The Workflow: Production latency spikes at 2 AM. The "SRE Agent" detects the anomaly, identifies a noisy neighbor, drains the node, and cordons it. It verifies stability and posts a post-mortem to Slack. You are only paged if the agent fails to solve the problem.‍
• The Shift: Human-out-of-the-Loop for standard operations. You manage the policy and the goals, not the tasks.

The Architect's Challenge: Managing Probabilistic Ops

This shift introduces a terrifying concept for traditional Ops people: Probabilistic Infrastructure.

Scripts are safe because they are deterministic; run them twice, get the same result. Agents are probabilistic; they might choose a slightly different path to solve a problem based on the context. As Solution Architects, how do we sleep at night knowing an AI has root access?

We design guardrails.

1. Constitutional AI (Policy-as-Code): Before any tool is executed, the agent's plan must pass through a deterministic policy engine (like Open Policy Agent). If an agent tries to execute terraform destroy on a production database, the policy engine kills the command hard, regardless of what the LLM "thinks" is right.‍
2. Contextual Permissions: We must architect agents with distinct permission boundaries. A "Diagnosis Agent" should have extensive Read permissions but zero Write permissions. A "Remediation Agent" should have Write permissions scoped strictly to the namespace it is repairing.‍
3. The Black Box Recorder: Every "thought" (reasoning step) and every "action" (CLI command) must be logged to a tamper-proof ledger. In a post-mortem, we need to be able to replay the agent's decision tree to understand why it made a specific choice.

Conclusion

The fear that AI will replace DevOps engineers is misplaced. It won't replace us; it will promote us.

In the future, you won't be paid to write the script that restarts the server. You will be paid to design the Cognitive Architecture that manages the fleet. You are becoming the Commander, orchestrating a team of silicon-based SREs that never sleep, never tire, and can react at the speed of light.

The future of DevOps isn't just code; it's Cognition.

As modern organizations continue their shift toward microservices, distributed systems, and high-velocity software delivery, NoSQL databases have become strategic building blocks. Their schema flexibility, scalability, and high throughput empower developers to move rapidly - but they also introduce operational, governance, and compliance risks. Without structured database change control, even a small update to a NoSQL document, key-value pair, or column family can cascade into production instability, data inconsistency, or compliance violations.

To sustain innovation at scale, enterprises need disciplined database change control for NoSQL - not as a bottleneck, but as an enabler of secure and reliable application delivery.

The Hidden Risks of Uncontrolled NoSQL Changes

Unlike relational systems, NoSQL databases place schema flexibility in the hands of developers. And the enterprises that rely on such NoSQL Database at scale are discovering the following truths:

• Flexibility without governance leads to instability.
• Data models must evolve as safely as application code.
• Compliance cannot rely on manual best-effort processes.

With structured change control:

• Schemas are versioned and peer-reviewed in Git
• Rollbacks are deterministic
• Environments stay consistent
• Audits pass without firefighting
• Data governance policies enforce themselves
• Compliance requirements (including GDPR’s “data integrity and confidentiality” mandate) are automatically met

NoSQL’s agility remains intact but reliability, safety, and traceability are added.

Database Change Control as Part of CI/CD

To eliminate risk and release bottlenecks, NoSQL change control needs to operate inside CI/CD pipelines - not outside them. This ensures that:

• Database updates are stored in Git as the system of record
• Pull requests enforce approvals and peer review
• Pipeline-driven testing validates the impact of schema changes before deployment
• Deployment logs provide traceability for governance and audit teams

A database change ceases to be a manual, tribal-knowledge activity and becomes a first-class software artifact - designed, tested, versioned, deployed, and rolled back automatically.

How Harness Safeguards NoSQL Change Delivery

Harness Database DevOps extends CI/CD best practices to NoSQL by providing automated delivery, versioning, governance, and observability across the entire change lifecycle, including MongoDB. Instead of treating database changes as a separate operational track, Harness unifies database evolution with modern engineering practices:

• DataMigration-as-Code stored in Git
• Automated verification before deployment
• Impact analysis and data preview
• Pipeline-level enforcement across every stage
• End-to-end audit trails and compliance logging
• Governed rollbacks and non-destructive deployments

This unification allows enterprises to move fast and maintain control, without rewriting how teams work. 

The Competitive Advantage of Doing This Right

High-growth teams that adopt change control for NoSQL environments report:

• Greater deployment confidence with lower production incident rates
• Sustained release velocity - without sacrificing data quality or security
• Reduced operational burden associated with GDPR, auditing, and governance
• Better alignment across developers, DBAs, SREs, and platform engineering

In short, the combination of NoSQL flexibility and automated governance allows enterprises to scale without trading speed for stability.

Final Thoughts

NoSQL databases have become fundamental to modern application architectures, but flexibility without control introduces operational risk. Implementing structured database change control - supported by CI/CD automation, runtime policy enforcement, and data governance - ensures that NoSQL deployments remain safe, compliant, and resilient even at scale.

Harness Database DevOps provides a unified platform for automating change delivery, enforcing compliance dynamically, and securing the complete database lifecycle - without slowing down development teams.

Welcome back to the quarterly update series! Catch up on the latest Harness 

Continuous Delivery innovations and enhancements with this quarter's Q4 2025 release. For full context, check out our previous updates:

• Q1 2025 Product Update: All the Latest Features Delivered by Harness
• Unlocking Innovation: Harness Q2 Feature Releases That Accelerate Your DevOps Journey
• Harness Q3 2025 update on continuous delivery enhancements

Q4 2025 builds on last quarter's foundation of performance, observability, and governance with "big-swing" platform upgrades that make shipping across VMs/Kubernetes safer, streamline artifacts/secrets, and scale GitOps without operational drag.

Deployments

Google Cloud Managed Instance Groups (MIG)

Harness now supports deploying and managing Google Cloud Managed Instance Groups (MIGs), bringing a modern CD experience to VM-based workloads. Instead of stitching together instance templates, backend services, and cutovers in the GCP console, you can run repeatable pipelines that handle the full deployment lifecycle—deploy, validate, and recover quickly when something goes wrong.

For teams that want progressive delivery, Harness also supports blue-green deployments with Cloud Service Mesh traffic management. Traffic can be shifted between stable and stage environments using HTTPRoute or GRPCRoute, enabling controlled rollouts like 10% → 50% → 100% with checkpoints along the way. After the initial setup, Harness keeps the stable/stage model consistent and can swap roles once the new version is fully promoted, so you’re not re-planning the mechanics every release.
Learn more about MIG

Multi-account AWS CDK deployments

AWS CDK deployments can now target multiple AWS accounts using a single connector by overriding the region and assuming a step-level IAM role. This is a big quality-of-life improvement for orgs that separate “build” and “run” accounts or segment by business unit, but still want one standardized connector strategy.
Learn more about Multi-account AWS CDK deployments

Automated ECS blue-green traffic shifting.

ECS blue-green deployments now auto-discover the correct stage target group when it’s not provided, selecting the target group with 0% traffic and failing fast if weights are ambiguous. This reduces the blast radius of a very real operational footgun: accidentally deploying into (and modifying) the live production target group during a blue/green cycle.
Learn more about automated ECS blue-green shifting

Azure WebApp API rate-limit resiliency

Improved resiliency for Azure WebApp workflows impacted by API rate limits, reducing flaky behavior and improving overall deployment stability in environments that hit throttling.

Harness Artifact Registry as a native source

HAR is now supported as a native artifact source for all CD deployment types except Helm, covering both container images and packaged artifacts (Maven, npm, NuGet, and generic). Artifact storage and consumption can live inside the same platform that orchestrates the deployment, which simplifies governance and reduces integration sprawl.

Because HAR is natively integrated, you don’t need a separate connector just to pull artifacts. Teams can standardize how artifacts are referenced, keep tagging/digest strategies consistent, and drive more predictable “what exactly are we deploying?” audits across environments.
Learn more about Harness Artifact Registry

GCP connector for Terraform steps

‍Terraform steps now support authenticating to GCP using GCP connector credentials, including Manual Credentials, Inherit From Delegate, and OIDC authentication methods. This makes it much easier to run consistent IaC workflows across projects without bespoke credential handling in every pipeline.

Learn more about GCP connector

AWS connector: AssumeRole session duration

AWS connectors now support configuring AssumeRole session duration for cross-account access. This is vital when you have longer-running operations (large Terraform applies, multi-region deployments, or complex blue/green flows) and want the session window to match reality.
Learn more about AWS connector

Terragrunt v1.x support (0.78.0+)

Harness now supports Terragrunt 0.78.0+ (including the v1.x line), with automatic detection and the correct command formats. If you’ve been waiting to upgrade Terragrunt without breaking pipeline behavior, this closes a major gap.
Learn more about Terragrunt

HashiCorp Vault JWT claim enhancements

Vault JWT auth now includes richer claims such as pipeline, connector, service, environment, and environment type identifiers. This enables more granular Vault policies, better secret isolation between environments, and cleaner multi-tenant setups.

‍

Continuous Verification (CV)

Custom webhook notifications for verification sub-tasks

CV now supports custom webhook notifications for verification sub-tasks, sending real-time updates for data collection and analysis (with correlation IDs) and allowing delivery via Platform or Delegate. This is a strong building block for teams that want deeper automation around verification outcomes and richer external observability workflows.
Learn more about Custom webhook notifications

Cross-project GCP Operations health sources

You can now query metrics and logs from a different GCP project than the connector’s default by specifying a project ID. This reduces connector sprawl in multi-project organizations and keeps monitoring setups aligned with how GCP estates are actually structured.
Learn more about cross-project GCP Operations

‍

Pipelines

This quarter’s pipeline updates focused on making executions easier to monitor, triggers more resilient, and dynamic pipeline patterns more production-ready. If you manage a large pipeline estate (or rely heavily on PR-driven automation), these changes reduce operational blind spots and help pipelines keep moving even when parts of the system don't function as expected.

Pipeline Notifications for “Waiting on User Action”

Pipelines can now emit a dedicated notification event when execution pauses for approvals, manual interventions, or file uploads. This makes “human-in-the-loop” gates visible in the same places you already monitor pipeline health, and helps teams avoid pipelines silently idling until someone notices.

Bitbucket Cloud Connector – Workspace API Token Support

Harness now supports Bitbucket Cloud Workspace API Tokens in the native connector experience. This is especially useful for teams moving off deprecated app password flows and looking for an authentication model that’s easier to govern and rotate.

Learn more about Bitbucket Cloud Connector

Pipeline Metadata Export to Data Platform (Knowledge Graph Enablement)

Pipeline metadata is now exported to the data platform, enabling knowledge graph style use cases and richer cross-entity insights. This lays the foundation for answering questions like “what deploys what,” “where is this template used,” and “which pipelines are affected if we change this shared component.”

Dynamic InputSet Branch Resolution for PR Triggers

Pull request triggers can now load InputSets from the source branch of the pull request. This is a big unlock for teams that keep pipeline definitions and trigger/config repositories decoupled, or that evolve InputSets alongside code changes in feature branches.

Learn more about Dynamic InputSet Branch Resolutions

Improved Trigger Processing Resilience

Trigger processing is now fault-tolerant; a failure in one trigger’s evaluation no longer blocks other triggers in the same processing flow. This improves reliability during noisy event bursts and prevents one faulty trigger from suppressing otherwise valid automations.

API Support for “Referenced By” Information on CD Objects

Added API visibility into “Referenced By” relationships for CD objects, making it easier to track template adoption and understand downstream impact. This is particularly useful for platform teams that maintain shared templates and need to measure usage, plan migrations, or audit dependencies across orgs and projects.

Detection & Recovery for Stuck Pipeline Executions

Harness now includes detection and recovery mechanisms for pipeline executions that get stuck, reducing reliance on manual support intervention. The end result is fewer long-running “zombie” executions and better overall system reliability for critical delivery workflows.

Dynamic Stages with Git-backed Pipeline YAMLs

Dynamic Stages can now source pipeline YAML directly from Git, with support for connector, branch, file path, and optional commit pinning. Since these values can be expression-driven and resolved at runtime, teams can implement powerful patterns like environment-specific stage composition, governed reuse of centrally managed YAML, and safer rollouts via pinned versions.

Learn more about Dynamic Stages‍

‍
GitOps

ApplicationSets as first-class entities

ApplicationSets are built for a problem every GitOps team eventually hits: one app template, dozens or hundreds of targets. Instead of managing a growing pile of near-duplicate GitOps applications, ApplicationSets act like an application factory—one template plus generators that produce the child applications.

With first-class ApplicationSet support, Harness adds an enhanced UI wizard and deeper platform integration. That includes Service/Environment integration (via standard labels), better RBAC alignment, validation/preview of manifests, and a cleaner operational experience for creating and managing ApplicationSets over time.
Learn more about ApplicationSets

Harness secret expressions in K8s manifests

You can now use Harness secret expressions directly inside Kubernetes manifests in a GitOps flow using the Harness Argo CD Config Management Plugin. The key shift is where resolution happens: secrets are resolved during Argo CD’s manifest rendering phase, which supports a pure GitOps pattern without requiring a Harness Delegate to decrypt secrets.

The developer experience is straightforward. You reference secrets using expressions like <+secrets.getValue("...")>, commit the manifest, and the plugin injects resolved values as Argo CD renders the manifests for deployment.
Learn more about Harness secret expressions

Argo Rollouts support

‍Harness GitOps now supports Argo Rollouts, unlocking advanced progressive delivery strategies like canary and blue/green with rollout-specific controls. For teams that want more than “sync and hope,” this adds a structured mechanism to shift traffic gradually, validate behavior, and roll back based on defined criteria.

This pairs naturally with pipeline orchestration. You can combine rollouts with approvals and monitoring gates to enforce consistency in how progressive delivery is executed across services and environments.
Learn more about Argo Rollouts support

‍

Want to try these GitOps capabilities hands-on?

‍Check out the GitOps Samples repo for ready-to-run examples you can fork, deploy, and adapt to your own workflows.
Explore GitOps-Samples

Next steps

That wraps up our Q4 2025 Continuous Delivery update. Across CD, Continuous Verification, Pipelines, and GitOps, the theme this quarter was simple: make releases safer by default, reduce operational overhead, and help teams scale delivery without scaling complexity.

If you want to dive deeper, check the “Learn more” links throughout this post and the documentation they point to. We’d also love to hear what’s working (and what you want next); share feedback in your usual channels or reach out through Harness Support.

‍

In most teams, the question is no longer "Do we need an internal developer portal?" but "Do we really want to run backstage ourselves?"

Backstage proved the internal developer portal (IDP) pattern, and it works. It gives you a flexible framework, plugins, and a central place for services and docs. It also gives you a long-term commitment: owning a React/TypeScript application, managing plugins, chasing upgrades, and justifying a dedicated platform squad to keep it all usable.

That's why there are Backstage alternatives like Harness IDP and managed Backstage services. It's also why so many platform teams are taking a long time to look at them before making a decision.

Why Teams Start Searching For Backstage Alternatives

Backstage was created by Spotify to fix real problems with platform engineering, such as problems with onboarding, scattered documentation, unclear ownership, and not having clear paths for new services. There was a clear goal when Spotify made Backstage open source in 2020. The main value props are good: a software catalog, templates for new services, and a place to put all the tools you need to work together.

The problem is not the concept. It is the operating model. Backstage is a framework, not a product. If you adopt it, you are committing to:

• Running and scaling the portal as a first-class internal product.
• Owning plugin selection, security reviews, and lifecycle management.
• Maintaining a consistent UX as more teams and use cases pile in.

Once Backstage moves beyond a proof of concept, it takes a lot of engineering work to keep it reliable, secure, and up to date. Many companies don't realize how much work it takes. At the same time, platforms like Harness are showing that you don't have to build everything yourself to get good results from a portal. 

When you look at how Harness connects IDP to CI, CD, IaC Management, and AI-powered workflows, you start to see an alternate model: treat the portal as a product you adopt, then spend platform engineering energy on standards, golden paths, and self-service workflows instead of plumbing.

The Three Real Paths: Build, Buy, Or Go Hybrid

When you strip away branding, almost every Backstage alternative fits one of three patterns. The differences are in how much you own and how much you offload:

1. Build: Self-Hosted Backstage Or Fully DIY Portal

What This Actually Means

You fork or deploy OSS Backstage, install the plugins you need, and host it yourself. Or you build your own internal portal from scratch. Either way, you now own:

• The UI and UX.
• The plugin ecosystem and compatibility matrix.
• Security, upgrades, and infra.
• Roadmapping and feature decisions.

Backstage gives you the most flexibility because you can add your own custom plugins, model your internal world however you want, and connect it to any tool. If you're willing to put a lot of money into it, that freedom is very powerful.

Where It Breaks Down

In practice, that freedom has a price:

• You need a dedicated team (often several engineers) to keep the portal healthy as adoption grows.
• You own every design decision and every piece of technical debt, forever.
• Plugin sprawl becomes real, especially when different teams install different components for similar problems.
• Scaling governance, RBAC, and standards enforcement almost always requires custom code.

This path could still work. If you run a very large organization and want to make the portal a core product, you need to have strong React/TypeScript and platform skills, and you really want to be able to customize it however you want, building on Backstage is a good idea. Just remember that you are not choosing a tool; you are hiring people to work on a long-term project.

2. Hybrid: Managed Backstage

What This Actually Means

Managed Backstage providers run and host Backstage for you. You still get the framework and everything that goes with it, but you don't have to fix Kubernetes manifests at 2 a.m. or investigate upstream patch releases.

Vendor responsibilities typically include:

• Running the control plane and handling infra.
• Coordinating upgrades and security fixes.
• Creating a curated library of high-value plugins.

You get "Backstage without the server babysitting."

Where The Trade-Offs Show Up

You also inherit Backstage's structural limits:

• The data model and catalog schema still look like Backstage.
• UI and interaction patterns follow Backstage's rules, which may not fit every team's mental model.
• Deeply customized plugins or data models still require serious engineering work.

Hybrid works well if you have already standardized on Backstage concepts, want to keep the ecosystem, and simply refuse to run your own instance. If you're just starting out with IDPs and are still looking into things like golden paths, self-service workflows, and platform-managed scorecards, it might be helpful to compare hybrid Backstage to commercial IDPs that were made to be products from the start.

3. Buy: Commercial IDPs 

What This Actually Means

Commercial IDPs approach the space from the opposite angle. You do not start with a framework, you start with a product. You get a portal that ships with:

• A software catalog.
• Ownership and scorecards.
• Self-service workflows.
• RBAC and governance tools.

The main point that sets them apart is how well that portal is connected to the systems that your developers use every day. Some products act as a metadata hub, bringing together information from your current tools. Harness does things differently. The IDP is built right on top of a software delivery platform that already has CI, CD, IaC Management, Feature Flags, and more.

Why Teams Go This Route

Teams that choose commercial Backstage alternatives tend to prioritize:

• Time to value in weeks, not quarters.
• Predictable total cost of ownership instead of wandering portal roadmaps.
• Built-in governance and security rather than "we'll build RBAC later."
• A real customer success partnership and roadmap, as opposed to depending on open-source momentum.

You trade some of Backstage's absolute freedom for a more focused, maintainable platform. For most organizations, that is a win.

Open Source Backstage Vs. Commercial Backstage Alternatives: Real Trade-Offs

People often think that the difference is "Backstage is free; commercial IDPs are expensive." In reality, the choice is "Where do you want to spend?"

When you use open source, you save money but lose engineering capacity. With commercial IDPs like Harness, you do the opposite: you pay to keep developers focused on the platform and save time. A platform's main purpose is to serve the teams that build on it. Who does the hard work depends on whether you build or buy.

This is how it works in practice:

Backstage is a solid choice if you explicitly want to own design, UX, and technical debt. Just be honest about how much that will cost over the next three to five years.

Commercial IDPs like Harness come with pre-made catalogs, scorecards, workflows, and governance that show you the best ways to do things. In short, it's ready to use right away. You get faster rollout of golden paths, self-service workflows, and environment management, as well as predictable roadmaps and vendor support.

The real question is what you want your platform team to do: shipping features in your portal framework, or defining and evolving the standards that drive better software delivery.

Where Commercial IDPs Fit Among Backstage Alternatives

When compared to other Backstage options, Harness IDP is best understood as a platform-based choice rather than a separate portal. It runs on Backstage where it makes sense (for example, to use the plugin ecosystem), but it is packaged as a curated product that sits on top of the Harness Software Delivery Platform as a whole.

There are a few design principles stand out:

1. Start from a product, not a bare framework. Backstage is intentionally a framework. Harness IDP is shipped as a product. Teams can start using the software right away because it already has a software catalog, scorecards, self-service workflows, RBAC, and policy-as-code. You add to it and shape it, but you don't put the basics together so that anyone can use it.
2. Make governance a first-class concern. Harness bakes environment-aware RBAC, policy-as-code (OPA), approvals, freeze windows, audit trails, and standards enforcement into the platform. Instead of adding custom plugins later, governance and security are built in from the start.
3. Prioritize actionability over passive visibility. Harness IDP does not stop at showing data. Because it runs directly over Harness CI, CD, IaC Management, Feature Flags, and related capabilities, it can drive workflows: spinning up new services from golden paths, managing environments, shutting down ephemeral resources, and wiring in repeatable self-service runbooks. The result is a portal that behaves more like an operational control plane.
4. Use AI where it can safely take action. The Harness Knowledge Agent is based on real delivery data, such as services, pipelines, environments, and scorecards. It can answer questions about who owns what and what happened in the past. It can also suggest or start safe actions under governance controls. That is not the same as AI features that only give a brief overview of catalog entries.

When you think about Backstage alternatives in terms of "How much of this work do we want to own?" and "Should our portal be a UI or a control plane?" Harness naturally fits into the group that sees the IDP as part of a connected delivery platform rather than as a separate piece of infrastructure.

Migration Realities: Moving Off Backstage Is Not A Free Undo Button

A lot of teams say, "We'll start with Backstage, and if it gets too hard, we'll move to something else." That sounds safe on paper. In production, moving from Backstage gets harder over time.

Common points where things go wrong include:

1. Custom plugins and extensions: One of Backstage's best features is its plugin ecosystem. It also keeps teams together. Over time, you build up a lot of custom plugins, scaffolder actions, and UI panels that are closely linked to your internal systems. Moving those to a different portal often means rewriting them completely, checking for compatibility, and sometimes even refactoring them.‍
2. Catalog complexity: Backstage catalogs tend to grow into hundreds or thousands of catalog-info.yaml files, custom entity kinds, and annotations. Moving this to a commercial IDP means putting that structure into the new system's data model while keeping ownership, relationships, and rules for governance. Trust in the new portal is directly affected by an incomplete migration here.‍
3. Golden path and scaffolder differences: Your existing scaffolder templates are wired into specific CI/CD tools and habits. Moving them to Harness IDP usually means changing the templates so that they run Harness pipelines, Harness environments, and IaC workflows instead of jobs from outside. That refactor is usually worth it, but it is still a lot of work.‍
4. Developer UX and "who moved my cheese?": Developers get used to Backstage's interaction patterns and custom dashboards. Changing to a new IDP always causes problems with adoption. The only way to avoid a revolt is to run portals at the same time and slowly roll out new golden paths.‍
5. Parallel system complexity: Running Backstage next to a new portal uses up a lot of platform bandwidth and makes things confusing for users if timelines aren't clear. Commercial vendors like Harness can help with this by providing migration tools and hands-on help, but you still need to plan for a migration window, not just flipping a switch.

The point isn't "never choose Backstage." The point is that if you do, you should think of it as a strategic choice, not an experiment you can easily undo in a year.

How To Evaluate Backstage Alternatives With A Clear Head

Whether you are comparing Backstage alone, Backstage in a managed form, or commercial platforms like Harness, use a lens that goes beyond feature checklists. These seven questions will help you cut through the noise.

1. Time to first value‍
   
   • Can you deliver a useful portal (catalog plus a couple of golden paths) in weeks?
   • Who owns upgrades, patches, and production reliability?‍
2. Total cost of ownership‍
   
   • How many engineers will this realistically consume over 3 years?
     
   • Is that time spent on differentiated work or reinvention?‍
3. Governance and security maturity‍
   
   • Do you get RBAC, policy-as-code, approvals, and audit trails out of the box?
   • Can you express environment-aware rules without writing custom code for every edge case?‍
4. Data model and extensibility‍
   
   • How hard is it to model services, infra, teams, and dependencies in a way that reflects reality?
   • Can you evolve the model as your architecture and org change?‍
5. Automation and actionability‍
   
   • Does the portal only aggregate data, or can it drive workflows like service creation, environment provisioning, and deployment rollbacks?
   • How directly does it connect to your CI/CD, IaC, and incident tooling?‍
6. AI and "agentic" workflows‍
   
   • Is AI just summarizing what you already see on dashboards, or can it actually update environments, run pipelines, and enforce policies safely?
   • How well grounded is that AI in your real delivery platform versus a generic data lake?‍
7. Exit strategy and lock-in‍
   
   • If you have to move in three to five years, how portable are your catalogs, templates, and automation?
   • Are you comfortable tying your IDP to a broader platform (like Harness) to gain deeper integration and efficiency?

If a solution cannot give you concrete answers here, it is not the right Backstage alternative for you.

Why Harness IDP Belongs On Your Shortlist

Choosing among Backstage alternatives comes down to one question: what kind of work do you want your platform team to own?

Open source Backstage gives you maximum flexibility and maximum responsibility. Managed Backstage reduces ops burden but keeps you within Backstage's conventions. Commercial IDPs like Harness narrow the surface area you maintain and connect your portal directly to CI/CD, environments, and governance.

If you want fast time to value, built-in governance, and a portal that acts rather than just displays, connect with Harness.

We’ve all seen it happen. A DevOps initiative starts with high energy, but two years later, you’re left with a sprawl of "fragile agile" pipelines. Every team has built their own bespoke scripts, security checks are inconsistent (or non-existent), and maintaining the system feels like playing whack-a-mole.

This is where the industry is shifting from simple DevOps execution to Platform Engineering.

The goal of a modern platform team isn't to be a help desk that writes YAML files for developers. The goal is to architect a "Golden Path"—a standardized, pre-vetted route to production that is actually easier to use than the alternative. It reduces the cognitive load for developers while ensuring that organizational governance isn't just a policy document, but a reality baked into every commit.

In this post, I want to walk through the architecture of a Golden Standard Pipeline. We’re going to look beyond simple task automation and explore how to weave Governance, Security, and Supply Chain integrity into a unified workflow that stands the test of time.

The Architectural Blueprint

A Golden Standard Pipeline isn't defined by the tools you use—Harness, Gitlab, GitHub Actions—but by its layers of validation. It’s not enough to simply "build and deploy" anymore. We need to architect a system that establishes trust at every single stage.

I like to break this architecture down into four distinct domains:

1. Governance Domain: checking if we should run this pipeline before we even start.
2. Integration Domain (The Inner Loop): getting fast feedback to developers.
3. Trust Domain (Supply Chain): creating proof that the software is safe.
4. Delivery Domain (The Outer Loop): getting it to production reliably.

Visualizing the Flow

Layer 1: Governance as the First Gate

The Principle: Don't process what you can't approve.

In traditional pipelines, we often see compliance checks shoehorned in right before production deployment. This is painful. There is nothing worse than waiting 20 minutes for a build and test cycle, only to be told you can't deploy because you used a non-compliant base image.

In a Golden Standard architecture, we shift governance to Step Zero.

By implementing Policy as Code (using frameworks like OPA) at the very start of execution, we solve a few problems:

• Drift Prevention: Pipelines simply won't run if they’ve been hacked to bypass standard steps.
• Resource Efficiency: We don't waste expensive compute building artifacts that are doomed to fail compliance.
• Security Baseline: Unauthorized workflows are stopped dead before they can access secrets or internal networks.

Layer 2: Parallelized Security Orchestration

The Principle: Security must speed the developer up, not slow them down.

The "Inner Loop" is sacred ground. This is where developers live. If your security scanning adds friction or takes too long, developers will find a way to bypass it. To solve this, we rely on Parallel Orchestration.

Instead of running checks linearly (Lint → then SAST → then Secrets), we group "Code Smells," "Linting," and "Security Scanners" to run simultaneously.

This gives us a huge architectural advantage:

1. Reduced Latency: We squash the wall-clock time by running I/O heavy checks in parallel.
2. Cost Optimization: We only trigger expensive Unit Test runners after the cheap, fast security checks pass. There is zero value in running a heavy test suite on a codebase that contains a hardcoded API key.

Layer 3: The Trust Layer (Supply Chain Security)

The Principle: Prove the origin and ingredients of your software.

This is the biggest evolution we've seen in CI/CD recently. We need to stop treating the build artifact (Docker image/Binary) as a black box. Instead, we generate three critical pieces of metadata that travel with the artifact:

1. SBOM (Software Bill of Materials): Think of this as the ingredients list on a food packet. It’s a machine-readable inventory of every library inside your container. When the next Log4j happens, you don't need to scan the world; you just query your inventory.
2. SLSA Provenance: This is an unforgeable ID card for your build. It proves where the build happened, when it happened, and what inputs were used. This is your defense against tampering attacks (like SolarWinds).
3. Cryptographic Signing: Finally, we sign the artifact using a private key (via Cosign). This acts like a digital wax seal; if the image is modified by even one bit after the build, the seal breaks, and your cluster will refuse to run it.

Layer 4: Immutable Delivery

The Principle: Build once, deploy everywhere.

A common anti-pattern I see is rebuilding artifacts for different environments—building a "QA image" and then rebuilding a "Prod image" later. This introduces risk.

In the Golden Standard, the artifact generated and signed in Layer 3 is the exact same immutable object deployed to QA and Production. We use a Rolling Deployment strategy with an Approval Gate between environments. The production stage explicitly references the digest of the artifact verified in QA, ensuring zero drift.

The Capability Map

To successfully build this, your platform needs to provide specific capabilities mapped to these layers.

Future-Proofing Your Platform

Tools change. Jenkins, Harness, GitHub Actions—they all evolve. But the Architecture remains constant. If you adhere to these principles, you future-proof your organization:

1. Decouple Policy from Pipeline: Store your policies separately from your pipeline YAML. This lets you update security rules globally without needing a massive migration project to edit hundreds of pipelines.
2. Standardize Interfaces: Use standard formats for your metadata (SPDX for SBOMs, In-toto for attestations). This prevents vendor lock-in and ensures your data is portable.
3. Invest in "Shift Left" Culture: The best architecture in the world fails if developers see it as a hurdle. Position the Golden Pipeline as a product that solves developer pain points (like setting up environments or managing credentials) while silently enforcing security in the background.

Conclusion

Adopting a Golden Standard architecture transforms the CI/CD pipeline from a simple task runner into a governance engine. By abstracting security and compliance into these reusable layers, Platform Engineering teams can guarantee that every microservice—regardless of the language or framework—adheres to the organization's highest standards of trust.

If you’re delivering software in 2026, you’re caught in a swirl. AI-assisted coding is accelerating development. Cloud-native architectures are multiplying both microservices and the pipelines required to deliver them. And increasingly, it’s DevOps teams - not dedicated security teams - who need to catch vulnerabilities before they reach production. 

Bolting application security testing (AST) onto your pipelines kinda worked up until now, but with AI accelerating code velocity and cloud scaling complexity, this approach is breaking down. The problem isn't just integrating security tools—it’s the friction they create. Context switching between platforms, alert fatigue from noise, and slowing down pipelines to chase down false positives. Security still feels bolted on—an external gate rather than a native part of how you build and deliver software.

That's why we're bringing AST natively into the Harness platform. Today, we're excited to announce that Qwiet AI—the AI-powered SAST and SCA engine we acquired last year—is now available as Harness SAST and SCA (with 45-day free trial), with pre-configured security steps in Harness Security Testing Orchestration (STO) and full configuration and results visibility directly in the Harness UI. Security testing that feels like it belongs in your pipeline—because it does.

Introducing Harness SAST and SCA

Most AST solutions flood developers with thousands of findings—many of which are theoretical vulnerabilities in code paths that never actually execute in production. This creates alert fatigue and slows down pipelines while teams triage false positives. Harness takes a fundamentally different approach powered by AI and reachability analysis. Instead of flagging every potential vulnerability, we use our patented code property graph (CPG) analysis to understand how data flows through your application—identifying only the vulnerabilities that are actually reachable through execution paths in your code. This means:

• Fewer false positives, more actionable results—by focusing on reachable vulnerabilities, Harness dramatically reduces noise. Teams spend less time investigating theoretical risks and more time fixing issues that matter.
• Deep visibility into open source risk—Harness SCA goes beyond basic dependency scanning to identify reachable vulnerabilities in third-party libraries, showing you which CVEs in your dependencies are actually exploitable in your application's context.
• AI-powered remediation—Harness SAST doesn't just identify vulnerabilities—it helps fix them with specific, actionable fix recommendations that developers can apply directly, based on your application's architecture and the data flows that make the vulnerability exploitable. 

The result? Security findings that developers actually trust—and act on.

Why Pipeline-Native AST Matters 

There’s a dirty secret of application security: every AST tool can integrate with CI/CD. Every vendor claims they shift left. But application security programs still stall at ~20-30% pipeline coverage because the operational burden doesn’t scale. Manual configuration, the need to piece together findings across multiple vendors and tools, and the challenge of orchestrating the right security testing across 100s or 1000s of pipelines all contribute.

When security testing runs as a first-party capability inside your CI/CD platform, three things happen: 

1. Configuration becomes templated and reusable—you define security steps once and propagate them across pipelines at scale, eliminating the manual toil that kills AppSec coverage. 
2. Feedback loops collapse—findings surface in the same interface where developers manage builds, deployments, and rollbacks, removing the context-switching tax that causes security issues to languish. 
3. Orchestration becomes policy-driven—instead of configuring individual scanners, you define what security posture looks like and let the platform take the right actions based on pipeline-level outcomes. 

The result is security testing that actually operationalizes at the pace and scale of modern software delivery—where covering 80% of your pipelines is a matter of policy enforcement, not heroic manual effort.

Pipeline-Native SAST and SCA with Harness

Harness AST combines the accuracy and actionability of Qwiet AI’s scanners with the operationalization at scale on the Harness platform. 

Pre-Defined Security Steps

Along with API Security Testing, Harness SAST and SCA are now available as pre-defined security steps in Security Testing Orchestration. This eliminates the complex setup and configuration work typically required to integrate security testing tools with CI/CD pipelines, allowing you to add security tests in minutes rather than hours. Instead of spending hours configuring a SAST scanner with the right language runtimes, authentication tokens, and result parsers, simply add the 'Harness SAST' step to your pipeline and you're scanning. This standardized approach ensures consistent security coverage across all projects while removing the friction that often causes teams to skip or delay security testing in their CI/CD workflows.

Virtual Pipeline Builder

Having Harness SAST and SCA as pre-configured steps in the STO step library transforms pipeline creation into an intuitive visual workflow. Developers can simply drag and drop security testing steps directly into their pipeline stages in Harness's virtual pipeline builder, selecting from industry-leading scanners without writing YAML configurations, managing container images, or troubleshooting integrations. The visual interface automatically handles the underlying orchestration, allowing teams to see exactly where security gates fit in their deployment workflow and adjust them with simple parameter changes.

Understanding Pipeline Posture with STO

STO provides a unified view of all security findings, consolidating results from Harness SAST and SCA alongside any of the 50+ integrated partner scanners into a single dashboard. Rather than jumping between different tool interfaces or parsing scattered reports, teams can view all vulnerabilities for a specific pipeline or aggregate findings across multiple pipelines to understand their broader application security posture. 

But STO doesn't just aggregate findings—it provides the context developers need to act. For each vulnerability, you can see which pipeline introduced it, which deployment it affects, and what remediation Harness SAST recommends. You can also set exemption policies, track remediation over time, and understand your security posture across the entire application portfolio—all without leaving the Harness platform.

Viewing SAST and SCA Findings

Harness STO displays comprehensive details for every SAST and SCA finding directly in the Harness UI, eliminating the need to switch to external scanner dashboards or export reports. Teams can click into any vulnerability to access full context about the issue, its severity, affected files, and remediation guidance—all within their existing workflow.

For SAST findings, Harness visualizes the complete data flow for each vulnerability, showing the "source-to-sink" execution path that illustrates how untrusted data propagates through application logic and is ultimately used in a sensitive operation. This visual representation provides precise code-level context based on static analysis, helping developers understand not just where a vulnerability exists, but exactly how malicious input could flow through their application to create a security risk. By mapping the entire taint flow, developers can see each step in the vulnerable code path and identify the optimal point for implementing fixes.

Each finding includes AI-powered remediation guidance from Harness, which explains the vulnerability details, the security concept behind why it's dangerous, and specific steps to fix the issue in context. Rather than generic advice, Harness AI analyzes the specific code pattern and provides tailored recommendations that help developers understand both the immediate fix and the underlying security principle, accelerating remediation while improving the team's security knowledge over time.

Try It for Free Today

Ready to experience integrated SAST and SCA in your pipelines? Harness is offering STO customers a 45-day free trial to explore how native application security testing can transform your development workflow. You can add comprehensive code and dependency scanning to your existing pipelines using our visual pipeline builder, consolidate findings into a single dashboard, and leverage AI-powered remediation guidance—all without complex setup or additional infrastructure to manage.

Reach out to your account team to start your free trial today and see how Harness SAST and SCA eliminate the friction that traditionally keeps security testing out of CI/CD pipelines. 

Request a demo

Harness AI is starting 2026 by doubling down on what it does best: applying intelligent automation to the hardest “after code” problems, incidents, security, and test setup, with three new AI-powered capabilities. These updates continue the same theme as December: move faster, keep control, and let AI handle more of the tedious, error-prone work in your delivery and security pipelines.

What’s New in Harness AI:

• Human-aware incident analysis that correlates conversations with changes
• AI-driven API naming that reduces security noise
• Natural-language auth script generation for faster AST onboarding
• AppSec agent for querying security data and generating policies

Human-Aware Change Agent for AI SRE

Harness AI SRE now includes the Human-Aware Change Agent, an AI system that treats human insight as first-class operational data and connects it to the changes that actually break production. Instead of relying only on logs and metrics, it listens to real incident conversations in tools like Slack, Teams, and Zoom and turns those clues into structured signals.​

• The AI Scribe captures key decisions, timestamps, symptoms, and “right before this happened…” moments from live conversations, filtering out unrelated chatter.​
• The Change Agent uses these human signals to drive a change-centric investigation across deployments, feature flags, config, infra changes, and ITSM records, then produces evidence-backed hypotheses such as, “This checkout deployment changed retry behavior 12 minutes before the incident, and latency spiked immediately after.”​​

By unifying human observations with the software delivery knowledge graph and change intelligence, teams get a much faster path from “what are we seeing?” to “what changed?” to “what should we roll back or fix safely?” The result is shorter incidents, clearer ownership, and a teammate-like AI that reasons about both people and systems in real time.​ Learn more in the announcement blog post.

AI-Powered API Naming for Cleaner Security Signals

Effective application security starts with knowing what you actually have in production. Traditional API naming based on regex heuristics often leads to over-merged or under-merged API groups, noisy inventories, and false positives across detection workflows.​

This month, API naming in our Traceable product gets a major upgrade with AI-powered API semantics:

• API naming is now powered by LLMs that understand intent, behavior, and functional semantics, not just URL or path similarity. The result is more stable, meaningful API groupings that reflect how your services actually behave.​
• The LLM-driven results were baselined against custom naming rules from advanced users and achieved >98.7% average match in internal benchmarking.​
• With cleaner API groupings, teams see reduced false positives across vulnerability detection, AST, and runtime protection, and a less noisy API inventory that’s easier for security and platform teams to act on.​

For security leaders trying to tame API sprawl, this is a foundational improvement that boosts signal quality across the entire platform.​

AI-Based Auth Script Generation: Faster, Safer API Security Testing Setup

Authentication setup has been one of the most consistent sources of friction for application security testing. Manual scripting, validation cycles, and back-and-forths often create bottlenecks — and a broken auth script can quietly invalidate an entire scan run.​

To solve this, all API Security Testing customers now get AI-based Authentication Script Generation:

• Generate auth scripts by simply describing the scenario in natural language; AI produces a ready-to-use script in a few seconds, which you can refine, edit, or use as a base for existing scripts.​
• The feature works alongside existing flows, so teams can keep using form-based or code-based auth with identical behavior while layering in AI where it helps most.​​

The result is less time lost to brittle auth setup, faster onboarding for new apps, and fewer failed scans due to script errors.​

You can find implementation details and examples in the docs. 

Chat with AppSec Agent: Security Data, in Plain Language

Security and platform teams often know the question they want to ask: “Where is this component used?” “Which exemptions are still pending?” , but answering it requires hopping across dashboards and stitching together filters by hand.​

The new AppSec Agent makes this dramatically easier by letting you query AppSec data using natural language.

Here's what it does:

• In Harness STO, you can ask about security issues and exemptions, then drill into issue-level insights from STO results without manually navigating views or composing complex filters. Questions like “Approve all valid pending exemptions in this project of issue type secret” become a single prompt instead of a multi-step workflow.​
• In Harness SCS, you can query for artifacts, code repos, SBOMs, chain-of-custody, and compliance results, then even generate OPA policies with a single prompt to block components based on license risk or vulnerable packages. For example, “Create an OPA policy to block the deployment of components licensed under the GPL-3.0 license” or “Help me identify whether the chalk and xz-utils components are present in any of the artifacts in this project” are fully supported.​
• The AppSec Agent is available across all production environments and integrates directly with Harness Security Testing Orchestration (STO) and Software Supply Chain Assurance (SCS).

This is a big step toward making AppSec data as queryable and collaborative as the rest of your engineering stack. Learn more in the docs.​

How This Fits the Harness AI Vision

Harness AI is focused on everything after code is written — building, testing, deploying, securing, and optimizing software through intelligent automation and agentic workflows. January’s updates extend that vision across:

• Security and AppSec: higher-fidelity API grouping, fewer false positives, and faster AST onboarding with AI-generated auth.
• SRE and Operations: human-aware incident response that unifies human and machine signals into a single, change-driven flow.
• Governance and Compliance: consistent with December’s AI governance updates, all of these capabilities inherit Harness’s approach of policy-aware AI, auditability, and RBAC-aligned actions.

Teams adopting these features can ship changes faster, investigate less, and focus more of their time on the work that actually moves the business — while Harness AI quietly handles the complexity in the background.

Kubernetes is a powerhouse of modern infrastructure — elastic, resilient, and beautifully abstracted. It lets you scale with ease, roll out deployments seamlessly, and sleep at night knowing your apps are self-healing.

But if you’re not careful, it can also silently drain your cloud budget.

In most teams, cost comes as an afterthought — only noticed when the monthly cloud bill starts to resemble a phone number. The truth is simple:

Kubernetes isn’t expensive by default.

Inefficient scheduling decisions are.

These inefficiencies don’t come from massive architectural mistakes. It’s the small, hidden inefficiencies — configuration-level choices — that pile up into significant cloud waste.

In this post, let’s unpack the hidden costs lurking in your Kubernetes clusters and how you can take control using smarter scheduling, bin packing, right-sizing, and better node selection.

The Hidden Costs Nobody Talks About

Over-Provisioned Requests and Limits

Most teams play it safe by over-provisioning resource requests — sometimes doubling or tripling what the workload needs. This leads to wasted CPU and memory that sit idle, but still costs money because the scheduler reserves them.

Your cluster is “full” — but your nodes are barely sweating.

Low Bin-Packing Efficiency

Kubernetes’s default scheduler optimizes for availability and spreading, not cost. As a result, workloads are often spread across more nodes than necessary. This leads to fragmented resource usage, like:

• A node with 2 free cores that no pod can “fit” into
• Nodes stuck at 5–10% utilization because of a single oversized pod
• Non-evictable pods holding on to almost empty nodes

Wrong Node Choices (Intel vs AMD, Spot vs On-Demand)

Choosing the wrong instance type can be surprisingly expensive:

• AMD-based nodes are 20–30% cheaper in many clouds
• Spot instances can cut costs dramatically for stateless workloads up to 70%
• ARM (e.g., Graviton in AWS) can offer up to 40% savings

But without node affinity, taints, or custom scheduling, workloads might not land where they should.

Zombie Workloads and Forgotten Jobs

Old cron jobs, demo deployments, and failed jobs that never got cleaned up — they all add up. Worse, they might be on expensive nodes or keeping the autoscaler from scaling down.

Node Pool Fragmentation

Mixing too many node types across zones, architectures, or families without careful coordination leads to bin-packing failure. A pod that fits only one node type can prevent the scale-down of others, leading to stranded resources.

Always-On Clusters and Idle Infrastructure

Many Kubernetes environments run 24/7 by default, even when there is little or no real activity. Development clusters, staging environments, and non-critical workloads often sit idle for large portions of the day, quietly accumulating cost.

This is one of the most overlooked cost traps.
Even a well-sized cluster becomes expensive if it runs continuously while doing nothing.

Because this waste doesn’t show up as obvious inefficiency — no failed pods, no over-provisioned nodes — it often goes unnoticed until teams review monthly cloud bills. By then, the cost is already sunk.

Idle infrastructure is still infrastructure you pay for.

Smarter Scheduling: Cost-Aware Techniques

Kubernetes doesn’t natively optimize for cost, but you can make it.

Bin Packing with Intent: Taints, Affinity, and Custom Schedulers

Encourage consolidation by:

• Using taints and tolerations to isolate high-memory or GPU workloads
• Applying pod affinity/anti-affinity to co-locate or separate workloads
• Leveraging Cluster Orchestrator with Karpenter to intelligently place pods based on actual resource availability and cost
• Use Smart Placement Strategies to place non-evictable pods efficiently

In addition to affinity and anti-affinity, teams can use topology spread constraints to control the explicit distribution of pods across zones or nodes. While they’re often used for high availability, overly strict spread requirements can work against bin-packing and prevent efficient scale-down, making them another lever that needs cost-aware tuning.

Scheduled Scaledown of Idle Resources

All of us go through a state where all of our resources are running 24/7 but are barely getting used and racking up costs even when everything is idle.A tried and proved way to avoid this is to scale down these resources either based on schedules or based on idleness.

Harness CCM Kubernetes AutoStopping let’s you scale down your Kubernetes workloads, AutoScaling Groups, VMs and many more based on either their activity or based on Fixed schedules to save you from these idle costs.

Cluster Orchestrator can help you to scale down the entire cluster or specific Nodepools when they are not needed, based on schedules

Right-Sizing Workloads

It’s often shocking how many pods can run on half the resources they’re requesting. Instead of guessing resource requests:

• Try Cluster Orchestrator’s Vertical Pod Autoscaler (VPA) with a single click
• Use Prometheus metrics to measure actual usage
• Analyze reports from visibility tools

Leverage Spot, AMD, or ARM-based Nodes

Make architecture and pricing work in your favor:

• Use node selectors or affinity rules to schedule less critical workloads to Spot nodes. You can use Harness’s Cluster Orchestrator to run your workloads partially in Spot instances. Spot nodes are up to 90% cheaper compared to On-Demand nodes
• Prefer AMD or Graviton nodes for stateless or batch jobs
• Separate workloads by architecture to avoid mixed pools

Use Fewer, More Efficient Node Pools

Instead of 10 specialized pools, consider:

• Consolidating into fewer, well-utilized pools
• Using node-level bin-packing strategies via Karpenter or Cluster Orchestrator
• Tuning autoscaler thresholds to enable more aggressive scale-down

Invisible Decisions Are Expensive

One overlooked reason why Kubernetes cost optimization is hard is that most scaling decisions are opaque. Nodes appear and disappear, but teams rarely know why a particular scale-up or scale-down happened.

Was it CPU fragmentation? A pod affinity rule? A disruption budget? A cost constraint?

Without decision-level visibility, teams are forced to guess — and that makes cost optimization feel risky instead of intentional.

Cost-aware systems work best when they don’t just act, but explain. Clear event-level insights into why a node was added, removed, or preserved help teams build trust, validate policies, and iterate safely on optimization strategies.

Scheduled Scale-Down of Idle Resources

One of the most effective ways to eliminate idle cost is time- or activity-based scaling. Instead of keeping clusters and workloads always on, resources can be scaled down when they are not needed and restored only when activity resumes.

With Harness CCM Kubernetes AutoStopping, teams can automatically scale down Kubernetes workloads, Auto Scaling Groups, VMs, and other resources based on usage signals or fixed schedules. This removes idle spend without requiring manual intervention.

Cluster Orchestrator extends this concept to the cluster level. It enables scheduled scale-down of entire clusters or specific node pools, making it practical to turn off unused capacity during nights, weekends, or other predictable idle windows.

Sometimes, the biggest savings come from not running infrastructure at all when it isn’t needed.

Treat Cost Like a First-Class Metric

Cost is not just a financial problem. It’s an engineering challenge — and one that we, as developers, can tackle with the same tools we use for performance, resilience, and scalability.

Start small. Review a few workloads. Test new node types. Measure bin-packing efficiency weekly.

You don’t need to sacrifice performance — just be intentional with your cluster design.

Check out Cluster Orchestrator by Harness CCM today!

Kubernetes doesn’t have to be expensive — just smarter.

‍

GitOps has become the default model for deploying applications on Kubernetes. Tools like Argo CD have made it simple to declaratively define desired state, sync it to clusters, and gain confidence that what’s running matches what’s in Git.

And for a while, it works exceptionally well.

Most teams that adopt GitOps experience a familiar pattern: a successful pilot, strong early momentum, and growing trust in automated delivery. But as adoption spreads across more teams, environments, and clusters, cracks begin to form. Troubleshooting slows down. Governance becomes inconsistent. Delivery workflows sprawl across scripts and tools.

This is the point many teams describe as “Argo not scaling.”

In reality, they’ve hit what we call the Argo ceiling.

The Argo ceiling isn’t a flaw in Argo CD. It’s a predictable inflection point that appears when GitOps is asked to operate at scale without a control plane.

What Is the Argo Ceiling?

The Argo ceiling is the moment when GitOps delivery starts to lose cohesion as scale increases.

Argo CD is intentionally designed to be cluster-scoped. That design choice is one of its strengths: it keeps the system simple, reliable, and aligned with Kubernetes’ model. But as organizations grow, that same design introduces friction.

Teams move from:

• A small number of clusters to dozens (or hundreds)
• A handful of applications to thousands
• One platform team to many autonomous product teams

At that point, GitOps still works — but operating GitOps becomes harder. Visibility fragments. Orchestration logic leaks into scripts. Governance depends on human process instead of platform guarantees.

The Argo ceiling isn’t a hard limit. It’s the point where teams realize they need more structure around GitOps to keep moving forward.

The Symptoms Teams See at Scale

Fragmented Visibility

One of the first pain points teams encounter is visibility.

Argo CD provides excellent insight within a single cluster. But as environments multiply, troubleshooting often turns into dashboard hopping. Engineers find themselves logging into multiple Argo CD instances just to answer basic questions:

• Where is this application deployed?
• Which environment failed?
• Was this caused by the same change?

What teams usually want instead is:

• A single place to see deployment status across clusters
• The ability to correlate failures back to a specific change or release
• Faster root-cause analysis without switching contexts

Argo CD doesn’t try to be a global control plane, so this gap is expected. When teams start asking for cross-cluster visibility, it’s often the first sign they’ve hit the Argo ceiling.

Glue Code and Script Entropy

As GitOps adoption grows, orchestration gaps start to appear. Teams need to handle promotions, validations, approvals, notifications, and integrations with external systems.

In practice, many organizations fill these gaps with:

• Jenkins pipelines
• GitHub Actions
• Custom scripts glued together over time

These scripts usually start small and helpful. But as they grow, they begin to:

• Accumulate environment-specific logic
• Encode tribal knowledge
• Become difficult to change safely

This is a classic Argo ceiling symptom. Orchestration lives outside the platform instead of being modeled as a first-class, observable workflow. Over time, GitOps starts to feel less like a modern delivery model and more like scripted CI/CD from a decade ago.

Awkward Promotion Flows

Promotion is another area where teams feel friction.

Argo CD is excellent at syncing desired state, but it doesn’t model the full lifecycle of a release. As a result, promotions often involve:

• Manual pull requests
• Repo hopping between environments
• Ad-hoc approvals and checks

These steps slow delivery and increase cognitive load, especially as the number of applications and environments grows.

Secret Sprawl

Git is the source of truth in GitOps — but secrets don’t belong in Git.

At small scale, teams manage this tension with conventions and external secret stores. At larger scale, this often turns into a patchwork of approaches:

• Different secret managers per team
• Custom templating logic
• Inconsistent access controls

The result is secret sprawl and operational risk. Managing secrets becomes harder precisely when consistency matters most.

Difficult Audits

Finally, audits become painful.

Change records are scattered across Git repos, CI systems, approval tools, and human processes. Reconstructing who changed what, when, and why turns into a forensic exercise.

At this stage, compliance depends more on institutional memory than on reliable system guarantees.

What Not to Do When You Hit the Ceiling

When teams hit the Argo ceiling, the instinctive response is often to add more tooling:

• More scripts
• More pipelines
• More conventions
• More manual reviews

Unfortunately, this usually makes things worse.

The problem isn’t a lack of tools. It’s a lack of structure. Scaling GitOps requires rethinking how visibility, orchestration, and governance are handled — not piling on more glue code.

Principles for Scaling GitOps Correctly

Before introducing solutions, it’s worth stepping back and defining the principles that make GitOps sustainable at scale.

Centralize Control (Without Centralizing Ownership)

One of the biggest mistakes teams make is repeating the same logic in every Argo CD cluster.

Instead, control should be centralized:

• RBAC policies
• Governance rules
• Audit trails
• Global visibility

At the same time, application ownership remains decentralized. Teams still own their services and repositories — but the rules of the road are consistent everywhere.

Orchestrate, Don’t Script

GitOps should feel modern, not like scripted CI/CD.

Delivery is more than “sync succeeded.” Real workflows include:

• Pre- and post-deployment steps
• Validations and checks
• Approvals and notifications

These should be modeled as structured, observable workflows — not hidden inside scripts that only a few people understand.

Automate Guardrails

Many teams start enforcing rules through:

• PR reviews
• Documentation
• Manual approvals

That approach doesn’t scale.

In mature GitOps environments, guardrails are enforced automatically:

• Objective policy checks
• Repeatable enforcement
• Rules that run before, during, and after deployment

Git remains the source of truth, but compliance becomes a platform guarantee instead of a human responsibility.

Why GitOps Needs a Control Plane

These challenges point to a common conclusion: GitOps at scale needs a control plane.

Git excels at versioning desired state, but it doesn’t provide:

• Cross-cluster visibility
• Consistent governance
• End-to-end workflow orchestration

A control plane complements GitOps by sitting above individual clusters. It doesn’t replace Argo CD — it coordinates and governs it.

Harness as the GitOps Control Plane

Harness provides a control plane that allows teams to scale GitOps without losing control.

Unified Visibility

Harness gives teams a single place to see deployments across clusters and environments. Failures can be correlated back to the same change or release, dramatically reducing time to root cause.

Structured Orchestration

Instead of relying on scripts, Harness models delivery as structured workflows:

• Pre- and post-sync steps
• Promotions and approvals
• Notifications and integrations

This keeps orchestration visible, reusable, and safe to evolve over time.

AI-Assisted Deployment Verification

Kubernetes and Argo CD can tell you whether a deployment technically succeeded — but not whether the application is actually behaving correctly.

Harness customers use AI-assisted deployment verification to analyze metrics, logs, and signals automatically. Rather than relying on static thresholds or manual checks, the system evaluates real behavior and can trigger rollbacks when anomalies are detected.

This builds on ideas from progressive delivery (such as Argo Rollouts analysis) while making verification consistent and governable across teams and environments.

Solving the GitOps–Secrets Paradox

Harness GitOps Secret Expressions address the tension between GitOps and secret management:

• Secrets are created and stored securely in the Harness platform
• GitOps manifests reference secrets using expressions
• The GitOps agent resolves them at runtime
• Secrets never leave customer infrastructure

This keeps Git clean while making secret handling consistent and auditable.

‍

What’s Next

The Argo ceiling isn’t a failure of GitOps — it’s a sign of success.

Teams hit it when GitOps adoption grows faster than the systems around it. Argo CD remains a powerful foundation, but at scale it needs a control plane to provide visibility, orchestration, and governance.

GitOps doesn’t break at scale.
Unmanaged GitOps does.

Ready to move past the Argo ceiling? Watch the on-demand session to learn how teams scale GitOps with confidence.

‍

Have you ever watched a “temporary” Infrastructure as Code script quietly become mission-critical, undocumented, and owned by someone who left the company two years ago? We can all related to a similar scenario, if not infrastructure-specific, and this is usually the moment teams realise the build vs buy IaC decision was made by accident, not design.

As your teams grow from managing a handful of environments to orchestrating hundreds of workspaces across multiple clouds, the limits of homegrown IaC pipeline management show up fast. It starts as a few shell scripts wrapping OpenTofu or Terraform commands often evolves into a fragile web of CI jobs, custom glue code, and tribal knowledge that no one feels confident changing.

The real question is not whether you can build your own IaC solution. Most teams can. The question is what it costs you in velocity, governance, and reliability once the platform becomes business-critical.

The Appeal and Cost of Custom IaC Pipelines

Building a custom IaC solution feels empowering at first. You control every detail. You understand exactly how plan and apply flows work. You can tailor pipelines to your team’s preferences without waiting on vendors or abstractions.

For small teams with simple requirements, this works. A basic OpenTofu or Terraform pipeline in GitHub Actions or GitLab CI can handle plan-on-pull-request and apply-on-merge patterns just fine. Add a manual approval step and a notification, and you are operational.

The problem is that infrastructure rarely stays simple.

As usage grows, the cracks start to appear:

• Governance gaps: Who is allowed to apply changes to production? How are policies enforced consistently? What happens when someone bypasses the pipeline and runs apply locally?
• State and workspace sprawl: Managing dozens or hundreds of state files, backend configurations, and locking behaviour becomes a coordination problem, not a scripting one.
• Workflow inconsistency: Each team builds its own version of “the pipeline.” What starts as flexibility turns into a support burden when every repository behaves differently.
• Security and audit blind spots: Secrets handling, access controls, audit trails, and drift detection are rarely first-class concerns in homegrown tooling. They become reactive fixes after something goes wrong.

At this point, the build vs buy IaC question stops being technical and becomes strategic.

What an IaCM Platform Is Actually Solving

We cannot simply label our infrastructure as code management platform as “CI for Terraform.” It exists to standardise how infrastructure changes are proposed, reviewed, approved, and applied across teams.

Instead of every team reinventing the same patterns, an IaCM platform provides shared primitives that scale.

Consistent Workspace Lifecycles

Workspaces are treated as first-class entities. Plans, approvals, applies, and execution history are visible in one place. When something fails, you do not have to reconstruct context from CI logs and commit messages.

Enforced IaC Governance

IaC governance stops being a best-practice document and becomes part of the workflow. Policy checks run automatically. Risky changes are surfaced early. Approval gates are applied consistently based on impact, not convention.

This matters regardless of whether teams are using OpenTofu as their open-source baseline or maintaining existing Terraform pipelines.

Centralised Variables and Secrets

Managing environment-specific configuration across large numbers of workspaces is one of the fastest ways to introduce mistakes. IaCM platforms provide variable sets and secure secret handling so values are managed once and applied consistently.

Drift Detection Without Custom Glue

Infrastructure drift is inevitable. Manual console changes, provider behaviour, and external automation all contribute. An IaCM platform detects drift continuously and surfaces it clearly, without relying on scheduled scripts parsing CLI output.

Module and Provider Governance

Reusable modules are essential for scaling IaC, but unmanaged reuse creates risk. A built-in module and provider registry ensures teams use approved, versioned components and reduces duplication across the organisation.

Why Building This Yourself Rarely Scales

Most platform teams underestimate how much work lives beyond the initial pipeline.

You will eventually need:

• Role-based access control.
• Approval workflows that vary by environment.
• Audit logs that satisfy compliance reviews.
• Concurrency controls and workspace locking.
• Safe ways to evolve pipelines without breaking dozens of teams.

None of these are hard in isolation. Together, they represent a long-term maintenance commitment. Unless building IaC tooling is your product, this effort rarely delivers competitive advantage.

How Harness IaCM Changes the Build vs Buy Equation

Harness Infrastructure as Code Management (IaCM) is designed for teams that want control without rebuilding the same platform components over and over again.

It supports both OpenTofu and Terraform, allowing teams to standardise workflows even as tooling evolves. OpenTofu fits naturally as an open-source execution baseline for new workloads, while Terraform remains supported where existing investment makes sense.

Harness IaCM provides:

• Default plan and apply pipelines that work out of the box.
• Workspace templates to enforce consistent backend configuration and governance.
• Module and provider registries to manage reuse safely.
• Policy enforcement, security checks, and cost visibility built directly into every run.

Instead of writing and maintaining custom orchestration logic, teams focus on infrastructure design and delivery.

Drift detection, approvals, and audit trails are handled consistently across every workspace, without bespoke scripts or CI hacks.

Making a Deliberate Build vs Buy IaC Decision

The build vs buy IaC decision should be intentional, not accidental.

If your organisation has a genuine need to own every layer of its tooling and the capacity to maintain it long-term, building can be justified. For most teams, however, the operational overhead outweighs the benefits.

An IaCM platform provides faster time-to-value, stronger governance, and fewer failure modes as infrastructure scales.

Harness Infrastructure as Code Management enables teams to operationalise best practices for OpenTofu and Terraform without locking themselves into brittle, homegrown solutions.

The real question is not whether you can build this yourself. It is whether you want to be maintaining it when the platform becomes critical.

Explore Harness IaCM and move beyond fragile IaC pipelines.

The rapid adoption of AI is fundamentally reshaping the software development landscape, driving an unprecedented surge in code generation speed. However, this acceleration has created a significant challenge for security teams: the AI velocity paradox. This paradox describes a situation where the benefits of accelerated code generation are being "throttled by the SDLC processes downstream," such as security, testing, deployment, and compliance, which have not matured or automated at the same pace as AI has advanced the development process.

This gap is a recognized concern among industry leaders. In Harness’s latest State of AI in Software Engineering report, 48% of surveyed organizations worry that AI coding assistants introduce vulnerabilities, and 43% fear compliance issues stemming from untested, AI-generated code.

This blog post explores strategies for closing the widening gap and defending against the new attack surfaces created by AI tooling.

‍

Defining the AI Velocity Paradox in Security

The AI velocity paradox is most acutely manifested in security. The benefits gained from code generation are being slowed down by downstream SDLC processes, such as testing, deployment, security, and compliance. This is because these processes have not "matured or automated at the same pace as code generation has."

Every time a coding agent or AI agent writes code, it has the potential to expand the threat surface. This can happen if the AI spins up a new application component, such as a new API, or pulls in unvalidated open-source models or libraries. If deployed without proper testing and validation, these components "can really expand your threat surface."

The imbalance is stark: code generation is up to 25% faster, and 70% of developers are shipping more frequently, yet only 46% of security compliance workflows are automated.

‍

The Dual Risk: Vulnerabilities vs. Compliance

The Harness report revealed that 48% of respondents were concerned that AI coding assistance introduced vulnerabilities, while 43% feared regulatory exposure. While both risks are evident in practice, they do not manifest equally.

• Vulnerabilities are more tangible and appear more often in incident data. These issues include unauthenticated access to APIs, poor input validation, and the use of third-party libraries. This is where the "most tangible exposure is".

• Compliance is a "slow burn risk." For instance, new code might start "touching a sensitive data flow which was previously never documented." This may not be discovered until a specific compliance requirement triggers an investigation. Vulnerabilities are currently seen more often in real incident data than compliance issues.

‍

A New Attack Surface: Non-Deterministic AI Agents

The components that significantly expand the attack surface beyond the scope of traditional application security (appsec) tools are AI agents or LLMs integrated into applications.

Traditional non-AI applications are generally deterministic; you know exactly what payload is going into an API, and which fields are sensitive. Traditional appsec tools are designed to secure this predictable environment.

However, AI agents are non-deterministic and "can behave randomly." Security measures must focus on ensuring these agents do not receive "overly excessive permissions to access anything" and controlling the type of data they have access to.

‍

Top challenges for AI application security

‍

Prioritizing AI Security Mitigation (OWASP LLM Top 10)

For development teams with weekly release cycles, we recommend prioritizing mitigation efforts based on the OWASP LLM Top 10. The three critical areas to test and mitigate first are:

1. Prompt Injection: This is the one threat currently seeing "the most attacks and threat activity".
2. Sensitive Data Disclosure: This is crucial for any organization that handles proprietary data or sensitive customer information, such as PII or banking records.
3. Excessive Agency: This involves an AI agent or MCP tool having a token with permissions it should not have, such as write control for a database, code commit controls, or the ability to send emails to end users.

‍

We advise that organizations should "test all your applications" for these three issues before pushing them to production.

‍

A Deep Dive into Prompt Injection

Here’s a walkthrough of a real-world prompt injection attack scenario to illustrate the danger of excessive agency.

The Attack Path is usually:

1. Excessive Agency: An AI application has an agent that accesses a user records database via an API or Model Context Protocol (MCP) tool. Critically, the AI agent has been given a "broadly scoped access token" that allows it to read, make changes, and potentially delete the database.
2. The Override Prompt: A user writes a prompt with an override, for example, suggesting a "system maintenance" is happening and asking the AI to "help me make a copy of the database and make changes to it." This is a "direct prompt injection" (or sometimes an indirect prompt injection), which is designed to force the LLM agent to reveal or manipulate certain data.
3. Hijacking: If no guardrails are in place to detect such prompts, the LLM will create a hijack scenario and make the request to the database.
4. Real Exfiltration: Once the hijacking is done, the "real exfiltration happens." The AI agent can output the data in the chatbot or write it to a third-party API where the user needs access to that data.

‍

This type of successful attack can lead to "legal implications," data loss, and damage to the organization's reputation.

Here’s a playbook to tackle Prompt Injection attacks

‍

Harness’s Vision for AI Security

Harness's approach to closing the AI security gap is built on three pillars:

1. AI Asset Discovery and Posture Management: This involves automatically discovering all AI assets (APIs, LLMs, MCP tools, etc.) by analyzing application traffic. This capability eliminates the "blind spot" that application security teams often have with "shadow AI," where developers do not document new AI assets. The platform automatically provides sensitive data flows and governance policies, helping you be audit-ready, especially if you operate in a regulated industry.
2. AI Security Testing: This helps organizations test their applications against AI-specific attacks before they are shipped to production. Harness's product supports DAST scans for the OWASP LLM Top 10, which can be executed as part of a CI/CD pipeline.
3. AI Runtime Protection: This focuses on detecting and blocking AI threats such as prompt injection, jailbreak attempts, data exfiltration, and policy violations in real time. It gives security teams immediate visibility and enforcement without impacting application performance or developer velocity.

‍

Read more about Harness AI security in our blog post.

Looking Ahead: The Evolving Attack Landscape

Looking six to 12 months ahead, the biggest risks come from autonomous agents, deeper tool chaining, and multimodal orchestration. The game has changed from focusing on "AI code-based risk versus decision risk."

Security teams must focus on upgrading their security and testing capabilities to understand the decision risk, specifically "what kind of data is flowing out of the system and what kind of things are getting exposed." The key is to manage the non-deterministic nature of AI applications.

‍

To stay ahead, a phased maturity roadmap is recommended:

1. Start with visibility.
2. Move to testing.
3. Then, focus on runtime protection.

By focusing on automation, prioritizing the most critical threats, and adopting a platform that provides visibility, testing, and protection, organizations can manage the risks introduced by AI velocity and build resilient AI-native applications.

‍

Learn more about tackling the AI velocity paradox in security in this webinar.

‍