Open-source projects are a crucial aspect of software development, but their increasing size and complexity can result in longer build times. This can make it difficult to meet software release deadlines and carry out effective testing.
In this article, we will build Facebook’s Proxygen project on Harness CI module and draw a comparison of the build time with Github Actions (Proxygen’s Default CI Tool)
Facebook’s Proxygen project is a collection of C++ HTTP libraries including an easy-to-use HTTP server. In addition to HTTP/1.1, Proxygen (rhymes with "oxygen") supports SPDY/3 and SPDY/3.1. Proxygen is not designed to replace Apache or nginx — those projects focus on building extremely flexible HTTP servers written in C that offer good performance but almost overwhelming amounts of configurability. Instead, the focus is on building a high performance C++ HTTP framework with sensible defaults that includes both server and client code and that's easy to integrate into existing applications.
To set up the Proxygen project on Harness CI, fork the facebook/proxygen repository into your GitHub account and sign up for Harness CI if you haven't already. Once you're in the Continuous Integration module, follow these steps to create your pipeline:
It is important to note that the root folder ".harness" is required for the YAML path. For more information on connectors, refer to the Harness CI documentation.
Here’s the YAML to build the proxygen project:
Please modify the connectors in the pipeline YAML to use the ones you created, go to the Triggers tab of the newly created pipeline to confirm the creation of two triggers, including a Pull Request trigger and a Push trigger. In this case, the Pull Request trigger should be enabled.
To compare the build time on GitHub Actions, enable the GitHub Actions workflow from the Actions tab on GitHub (the cloned repository already has the actions workflow file).
Finally, create a Pull Request with a few source or test file changes. This will trigger the Harness CI pipeline, as well as the GitHub Actions workflow if enabled.
Here are the execution times for a same Pull Request triggered build on GitHub Actions and Harness CI.
The pipeline can consist of several stages, each of which is composed of one or more steps. In this case we have a single stage pipeline with the stage name as "Build" and has 5 steps: "Fetch Deps", "Build Deps", "Build Proxygen", “Copy Artifacts” and “Test Proxygen”.
The "Fetch Deps" step fetches dependencies needed for building the project using the "getdeps.py" script. The "Build Deps" step builds the dependencies, and the "Build Proxygen" step builds the project itself.
The next step i.e "Copy artifacts" copies the built artifacts to a final installation location. The final step is called "Test Proxygen" and runs the tests for the project.
The pipeline is defined to run on a Linux operating system and uses the AMD64 architecture on Harness Cloud infrastructure
This pipeline uses a matrix strategy, which allows for parallel execution of steps with different dependencies. In this pipeline, the matrix strategy is used to define a set of dependencies required for building the "proxygen" project, and to execute the "Fetch Deps" and "Build Deps" steps in parallel for each dependency.
The matrix strategy is implemented over the dependency list, which in this case includes dependencies such as "ninja", "cmake", "zlib", etc.. For each dependency in the list, a parallel execution of the "Fetch Deps" and "Build Deps" steps is initiated. This means that if there are 10 dependencies in the list, then there will be 20 parallel executions of the "Fetch Deps" and "Build Deps" steps, 10 for each step.
By using the matrix strategy the pipeline can make efficient use of resources by executing multiple steps in parallel which can significantly reduce the overall time required to complete the pipeline. This is particularly useful in cases where there are multiple configurations or dependencies that need to be built, as it allows for a more streamlined and efficient execution of the pipeline.
Harness CI is built for speed and offers various parallelism strategies as well as high speed Cloud infrastructure to dramatically improve the build times. Harness also offers several other build and test optimizations to shorten build times (and cost) while also improving developer productivity.
Sign up for free and try it yourself!