Bazel With Sonarqube Coverage

Software development happens at the speed of light. Thousands of code lines are added to the codebase every day, and we should always verify whether these lines of code are executing as part of application, or just lying in the source code.

To check whether new or modified code is executing, developers write unit tests. Unit tests make sure that new or modified code is working as expected, in both passing and failing conditions before they become part of the main repository.

What Is Bazel?

Bazel is a build tool developed by Google. Google utilizes this tool internally, where it is known as Blaze, to build their applications. This is the latest open-source build tool, and a lot of development is currently happening on it. But Bazel’s best feature is the incremental build, where Bazel identifies what was changed in the source code and builds only that part – it keeps the rest in its cache and simply reads from there. In other words, your first build will take time, but after that, every incremental build will take far less time – unless you cleared out your cache.

For more information on how to get started with Bazel, look here.

What Is Code Coverage?

As mentioned in the introduction, thousands of lines of code are added to the codebase every day. It’s essential to verify that all of the code is running as part of the application. Ideally, there shouldn’t be a single line of code that’s lying in the source code unused – that's not good coding practice.

Code coverage is a term used in the software industry to show how much of the code is executing and reachable in passing and failing situations, and then how much of the code is unreachable in any condition. Moreover, it reveals the effectiveness of your tests. This value can be generated by test cases, such as unit tests. Lines covered by executing test cases generate your coverage percentage. Almost all software companies put an 80% threshold minimum on code coverage. However, for some security and finance companies, this coverage percentage can go up to 100%. 

Let’s look at an example of what this percentage signifies. Suppose you have ten lines of code in your source file, and your code coverage minimum is 80%. This means that out of ten lines of code that you added/modified, only eight are reachable via test cases, and two aren’t reachable in any condition.

Why Is Code Coverage Necessary?

Code coverage should be calculated, as it gives you insights into issues in your source code, and how you can improve it. When it comes to large projects, increasing code coverage above 70-80% means that your application is maintaining code quality.

1. Coverage Insights: This reveals what percentage of code going into codebase is actually reachable, and on the basis of that unreachable code, can be identified and necessary action taken against it.

2. Vulnerabilities: Extra code that’s unreachable can be exploited by anyone, anytime, and in any way. So once you’ve identified the code that unnecessarily became part of the codebase, you can remove it to secure your application.

3. Test Cases: Less coverage numbers can’t only come from extra code. However, they can come from insufficient test cases. This means that developers can identify and write more test cases to cover the uncovered lines and bring the code coverage number closer to the defined threshold. Furthermore, this gives confidence on how the code will work in various situations.

4. Code Quality: Increasing code coverage helps spot problems. That gives teams confidence in their tests, and it increases code quality.

These are just some of the reasons to consider having code coverage. There are many other important reasons as well.

How Does Harness Calculate Code Coverage?

Many well-known tools on the market generate code coverage reports. Sonarqube is one of the industry’s most recognized tools, and Harness uses it. Sonarqube can generate multiple kinds of reports – not just on code coverage: code duplicity, code smells, and bug lists with level-of-risk. The following is a screenshot of the Sonar server report table.

Unfortunately there’s no direct integration between Bazel and Sonarqube as of now. Therefore, in Harness, we’re using third-party code—more of a parser—to integrate Bazel with Sonarqube.

Bazel has its own algorithm to generate coverage numbers by running test cases against the source code. In turn, Bazel generates the coverage report in the LCOV format, which isn’t the same as the Sonarqube format. Therefore, we’re using a parser to convert the LCOV formatted file to a format that Sonarqube understands. Then, we upload that file to the Sonarqube server to display various reports in a metrics-style table.

As of now, code coverage with Bazel and Sonarqube is an expensive and time-consuming operation. Therefore, in Harness, we’re running these coverage numbers at regular intervals—not as part of GitHub pull requests—and then sending reports to functional owners.

How Does the Bazel and Sonarqube Integration Work?

Bazel has its own algorithm to generate coverage numbers with the command "bazel coverage <Module Name>" in the LCOV format. This means that nothing extra needs to be written in the Bazel files. However, for integrating with Sonarqube, a couple of Bazel targets must be defined in the Bazel files. One target does all of the parsing operations and creates the files that Sonarqube understands, and the other sends the report to the Sonarqube server. The following steps are a high-level explanation for achieving this integration.

1. Create a folder in your codebase to store the third-party snippet mentioned earlier. Bazel will create a runnable target from these source files to complete the integration.

2. Create a Bazel target at the module level. I am assuming that you have a monorepo with multiple modules hosted in it. This target will fetch the information regarding your source files, test cases, and module name.

sq_project(
   name = "sq_mycomponent",
   project_key = "com.example.project:component",
   project_name = "My Project :: Component",
   srcs = [
       "//path/to/component:java_srcs",
   ],
   targets = [
       "//path/to/component:component",
   ],
   test_srcs = ["//path/to/component:java_test_srcs"],
   test_targets = [
       "//path/to/component:FirstComponentTest",
       "//path/to/component:SecondComponentTest",
   ],
   test_reports = ["//:test_reports"],
   tags = ["manual"],
   visibility = ["//visibility:public"],
)

3. Create a Bazel target at the root level of your project that contains the project name in the Sonar server and a list of modules as part of that project name.

sonarqube(
  name = "TargetName",
  srcs = [],
  coverage_report = ":coverage_report",
  modules = get_sonarqube_targets_seperated(),
  project_key = "portal project key",
  project_name = "portal project name",
  scm_info = [":git"],
  sq_properties_filename = "sany-filename.properties",
  tags = ["manual"],
  targets = [],
)

4. Finally, run "bazel run  <TargetName> -Dsonar.host.url=${SONAR_HOST_URL} -Dsonar.login=${SONAR_AUTH_TOKEN}" with a Sonar server DNS name or IP, along with an authentication token to send the report to the Sonar server.

Conclusion

Delivering software to customers requires many different components, and they all share the same goal – a successful and high-quality product. Code coverage alone can’t be considered as the metrics to decide code quality. However, it should be considered a vital part for achieving high-quality code. 

Considering code coverage on a daily basis keeps not only QA engineers aware of the code, but also developers from the test cases perspective. Moreover, it helps remove vulnerabilities, as well as keeps source code neat and clean with no extra or unwanted lines of code.

Want to keep reading? Learn about why we switched from Maven to Bazel in another article.