Test Coverage Reports For Maven Projects In SonarQube 8.3.x

calendarJun 10, 2020 · 5 min read · integration-testing jacoco maven sonarqube test-coverage test-report unit-testing  ·
Share on: linkedincopy

Some years ago I write a blog post about how to generate test reports in SonarQube separate in test report for unit tests and for integration tests. Since SonarQube 6.2 the test report isn't separate in these categories any more (see SonarQube's blog post). SonarQube merges all test reports to one test report with an overall coverage. So how to configure JaCoCo Maven Plugin if you have separate your tests in unit tests (running by Maven Surefire Plugin) and integration tests (running by Maven Failsafe Plugin) in your Maven project.

In the following sections, a solution is presented that meets following criteria:

  • Maven is used as build tool.
  • The project can be a multi module project.
  • Unit tests and integration tests are parts of each module.
  • Test coverage is measured by JaCoCo Maven Plugin.

The road map for the next section is that firstly the Maven project structure is shown for the separation of unit and integration tests. Then the Maven project configuration is shown for having separate unit test runs and integration test runs.  After that, we have a look on the Maven project configuration for the test report generation that covers unit and integration tests. At the end, SonarQube's configuration is shown for the test report visualization in the SonarQube's dashboard.

Maven Project Structure

At first, we look at how a default Maven project structure looks like for a single module project.

1my-app
2├── pom.xml
3├── src
4│   ├── main
5│   │   └── java
6│   └── test
7│       └── java

The directory src/main/java contains the production source code and the directory src/test/java contains the test source code. We could put unit tests and integration tests together in this directory. But we want to separate these two types of tests in separate directories. Therefore, we add a new directory called src/it/java. Then unit tests are put in the directory src/test/java and the integration tests are put in the directory src/it/java, so the new project structure looks like the following one.

1my-app
2├── pom.xml
3├── src
4│   ├── it
5│   │   └── java
6│   ├── main
7│   │   └── java
8│   └── test
9│       └── java

Unit And Integration Test Runs

Fortunately, the unit test run configuration is a part of the Maven default project configuration. Maven runs these tests automatically if following criteria are met:

  • The tests are in the directory src/test/java and
  • the test class name either starts with Test or ends with Test or TestCase.

Maven runs these tests during the Maven's build lifecylce phase test.

The integration test run configuration has to be done manually. It exists Maven plugins that can help. We want that the following criteria are met:

  • integration tests are stored in the directory src/it/java and
  • the integration test class name either starts IT or ends with IT or ITCase and
  • integrations tests runs during the Maven's build lifecycle phase integration-test.

Firstly, Maven has to know that it has to include the directory src/it/java to its test class path. Here, the Build Helper Maven Plugin can help. It adds the directory src/it/java to the test class path.

 1            <plugin>
 2                <groupId>org.codehaus.mojo</groupId>
 3                <artifactId>build-helper-maven-plugin</artifactId>
 4                <version>3.1.0</version>
 5                <executions>
 6                    <execution>
 7                        <goals>
 8                            <goal>add-test-source</goal>
 9                            <goal>add-test-resource</goal>
10                        </goals>
11                        <configuration>
12                            <sources>
13                                <source>src/it/java</source>
14                            </sources>
15                            <resources>
16                                <resource>
17                                    <directory>src/it/resources</directory>
18                                </resource>
19                            </resources>
20                        </configuration>
21                    </execution>
22                </executions>
23            </plugin>

The above code snippet has to be inserted into the section <project><build><plugins> in the project root pom.

Maven's build lifecycle contains a phase called integration-test. In this phase, we want to run the integration test. Fortunately, Maven Failsafe Plugins' goal integration-test is bind to this phase automatically when it's set up in the POM. If you want that the build fails when the integration tests fails then the goal verify also has to be added into the POM:

 1            <plugin>
 2                <groupId>org.apache.maven.plugins</groupId>
 3                <artifactId>maven-failsafe-plugin</artifactId>
 4                <version>3.0.0-M4</version>
 5                <configuration>
 6                    <encoding>${project.build.sourceEncoding}</encoding>
 7                </configuration>
 8                <executions>
 9                    <execution>
10                        <goals>
11                            <goal>integration-test</goal>
12                            <goal>verify</goal>
13                        </goals>
14                    </execution>
15                </executions>
16            </plugin>

Again, the above code snippet also has to be inserted into the section <project><build><plugins> in the project root pom. Then Maven Failsafe Plugin runs the integration tests automatically, when their class name either starts with IT or ends with IT or ITCase.

Test Report Generation

We want to use the JaCoCo Maven Plugin for the test report generation. It should generate test reports for the unit tests and for the integration tests. Therefore, the plugin has to two separated agents, that have to be prepared. Then they generate the report during the test runs. The Maven's build lifecycle contains own phases for preparation before the test phases (test and integration-test). The preparation phase for the test phase is called process-test-classes and the preparation phase for integration-test phase is called pre-integration-test. JaCoCo binds its agent to these phases automatically, when its goals prepare-agent and prepare-agent-integration are set up in the POM. But this is not enough. JaCoCo also has to create a report, so that SonarQube can read the reports for the visualization. Therefore, we have to add the goals report and report-integration in the POM:

 1            <plugin>
 2                <groupId>org.jacoco</groupId>
 3                <artifactId>jacoco-maven-plugin</artifactId>
 4                <version>0.8.5</version>
 5                <executions>    
 6                    <execution>
 7                        <goals>  
 8                            <goal>prepare-agent</goal>
 9                            <goal>prepare-agent-integration</goal>
10                            <goal>report</goal>
11                            <goal>report-integration</goal>
12                        </goals>  
13                    </execution>
14                </executions>  
15            </plugin>

Again, it is a part of the section <project><build><plugins>.

Now, we can run the goal mvn verify and our project is built inclusive unit and integration test and inclusive generating two test reports.

SonarQube Test Report Visualization

Now, we want to visualize our test reports in SonarQube. Therefore, we have to run the Sonar Maven 3 Plugin (command mvn sonar:sonar)  in our project after a successful build. So Sonar Maven Plugin knows where to upload the report, we have to configure our SonarQube instance in ~/.m2/setting.xml:

 1    <profile>
 2      <id>sonar</id>
 3      <activation>
 4        <activeByDefault>true</activeByDefault>
 5      </activation>
 6      <properties>
 7        <!-- Optional URL to server. Default value is http://localhost:9000 -->
 8        <sonar.host.url>http://localhost:9000</sonar.host.url>
 9      </properties>
10    </profile>

When we open our project in the SonarQube dashboard, we see the overall test coverage report.

Summary

This blog describes how to generate test reports for a Maven build if unit tests and integration tests run separately. On GitHub, I host a sample project that demonstrate all configuration steps. As technical environment I use

  • Maven 3.6.3
  • Maven Plugins:
    • Maven Surefire Plugin
    • Maven Failsafe Plugin
    • Build Helper Maven Plugin
    • Jacoco Maven Plugin
    • Sonar Maven Plugin
  • SonarQube 8.3.1
  • Java 11
  1. Jacoco Maven plugin project site
  2. Maven Failsafe Plugin  project site
  3. Build Helper Maven Plugin project site
  4. SonarQube documentation about Test Coverage in common
  5. A sample Maven project on GitHub