Run Ruby Builds Up To 4x Faster Using Harness CI Test Intelligence

Authors:

Table of Contents

Companies strive to deliver software of the highest quality at the lowest possible cost. The quality of every software project is determined by balancing time, resources, and scope, otherwise known as the project management triangle. Since these three factors are linked, delivering high quality software quickly and efficiently almost always means sacrificing scope (features).

The exception to this rule is innovation.

Harness Test Intelligence™ (TI) helps companies innovate by decreasing test execution time, without reducing quality, sacrificing features, or increasing costs.

Harness Test Intelligence, which debuted in October 2022, is one of the features that makes Harness CI up to 4x faster than other CI solutions. TI reduces test time by running only the subset of tests relevant to the code changes that triggered the pipeline. TI supports Java, Kotlin, Scala, C#, and today we are thrilled to announce that support for Ruby is currently in public preview.

Demo

Vagrant is a popular open source project with over 5,000 Ruby rspec tests. Check out this demo to see how TI reduces Vagrant’s Ruby test times over 90% compared to other CI providers.

In this post, we will give you an overview of TI, along with steps to reproduce the above demo in your own Harness account.

Where Test Intelligence Fits in Your SDLC

Testing is a vital component of every Continuous Integration pipeline. Testing ensures the quality of your application before code advances to the next phase in your SDLC. Running all tests on every code change is expensive and time-consuming. This leaves developers with the difficult decision of running all tests when introducing changes or to deferring tests until later in the development cycle.

Author Robert Galanakis says, “The fastest code is the code which does not run”, in the spirit of writing simple, maintainable code. We can extend this perspective to software testing: The fastest tests are the tests which do not run.

How Does Test Intelligence Work?

Using Machine Learning (ML), Test Intelligence examines changes to source code and tests to determine what tests should be run for a given change. When a code change is pushed to a repository, TI uses the following metrics to select tests:

Changed code: TI uses the list of files changed in the commit range to select tests that are associated directly or indirectly with the changes.

Changed tests: When a test is changed, TI selects the test for execution, even if the code related to the test hasn’t changed.

New tests: When new tests are added, TI finds any correlations between any new or existing code, and selects and runs those tests.

TI is always up to date and syncs when you merge code to any branch.

Try Ruby Test Intelligence Yourself

To reproduce the above demo in your Harness account, follow these steps.

Sign up for your free Harness account.
Follow this guide.
You will see a subset of tests run between the two stages in your pipeline. Instead of running all 5,000+ Ruby rspec tests, TI selects only the subset of tests relevant to the change.
In the log output, you can see the number of tests that were selected in each parallel stage.