How to setup JaCoCo code coverage with Maven & Gradle

Step by step configuration of the JaCoCo code coverage tool for Java project using Maven & Gradle build tools

First of all, code coverage tools indicate what percentage of your codebase is covered by your tests.

In this article, we will actually get our hands dirty and add code coverage to a java project that uses Maven or Gradle build tool.

In the following article, I have briefly explained 10 reasons why code coverage plays a crucial role to a product’s quality and success in the long run.

There are many tools out there for different projects, and we will focus on JaCoCo, which stands for JavaCodeCoverage tool.

Java code coverage tools

There are different code coverage tools for different languages, and I am going to focus on Java. The most famous code coverages tools are

1. JaCoCo — https://www.jacoco.org/jacoco/index.html
2. Cobertura — https://cobertura.github.io/cobertura/
3. Open Clover — https://openclover.org/
4. JCov — https://wiki.openjdk.java.net/display/CodeTools/jcov

The most recent release of JaCoCo was October 2019 compared to Cubertura, which is 2013.

Here is a comparison of code coverage tools by Atlassian in contrast to their product Atlassian Clover.

Based on my work experiences, JaCoCo seems to be the most commonly used open-source code coverage tool, and we are going to focus on that.

What is JaCoCo?

JaCoCo is an open-source code coverage tool that measures coverage of

• Instructions
• Lines
• Branches

JaCoCo can instrument Java bytecode via a java agent dynamically or static instrumentation, which is done upfront by changing the code that’s going to be tested in order to measure its coverage.

JaCoCo is one of the most active open-source code coverage tools that supports up to Java 14 with the most recent release in 2019.

JaCoCo also offers integrations with CI systems such as Jenkins as well as plugins for IDEs such as Eclipse and IntelliJ. Ther are Maven and Gradle based plugins for JaCoCo.

How to setup JaCoCo with Maven?

Here is what you need to setup JaCoCo in a Maven-based project. Let’s generate a maven-based spring boot project and then set it up with the JaCoCo code coverage tool.

Step 1. Generate the project

Navigate to https://start.spring.io/ to generate a project with an appropriate version of build tool (maven), language (Java), the default version of spring

Step 2. Import project into IDE (i.e., Eclipse)

The above would generate a zip demo.zip file, and you can extract it in your workspace. Let’s get started and I will use Eclipse IDE

cd demo 
mvn eclipse:clean 
mvn eclipse:eclipse

After running mvn eclipse:eclipse the project is ready to import to Eclipse. Right-click on Eclipse IDE’s project explorer window (on the left side) and select Import and from the window that opens use Existing Projects into Workspace from under General category as shown below

After clicking Next you will see the following window and use Browse... to select the root of your project from the workspace. Upon selection, if you have successfully run mvn eclipse:eclipse command then your project should get listed as shown below

Step 3. Add JaCoCo plugin to the pom.xml

Now that the project is added to your IDE, let’s modify the pom.xml to add the JaCoCo configuration.

As shown above, all that is needed to get the JaCoCo code coverage working is to add the JaCoCo plugin in the build section of pom.xml file of your project.

Step 4. Generate code coverage report

In order to generate a code coverage report, run the mvn test in the project and you will see the following lines relating to jacoco towards the end of the console

...
[INFO] --- jacoco-maven-plugin:0.8.5:report (report) @ demo ---
[INFO] Loading execution data file /Users/raf/workspace/demo/target/jacoco.exec
...

Since we have specified the JaCoCo plugin <phase>test</phase> that means during the test phase generate a report as per the <goal>report</goal> execution goal.

How to setup JaCoCo with Gradle?

Gradle is another alternative to the Maven build tool. Let’s configure JaCoCo in a Java project that uses Gradle.

Step 1. Generate the project

Navigate to https://start.spring.io/ to generate a project with an appropriate version of build tool (Gradle), language (Java), the default version of spring

Now click Generate to download the project.

Step 2. Import project into IDE (i.e., Eclipse)

Let’s update the project’s build.gradle file and under the plugins include eclipse as shown below

plugins {
        id 'org.springframework.boot' version '2.3.1.RELEASE'
        id 'io.spring.dependency-management' version '1.0.9.RELEASE'
        id 'java'
        id 'eclipse'
}

Now you can run the ./gradlew eclipse command in the project directory to generate the necessary Eclipse specific configuration files.

Next, let’s import the project as Existing Gradle Project as shown below

In the next screen, specify where the project is and keep clicking next until finally done, and the project appears in the project explorer window.

Step 3. Add the JaCoCo configs to build.gradle

Looking into https://docs.gradle.org/current/userguide/jacoco_plugin.html documentation, to get the JaCoCo setup going with Gradle we need to add the jacoco under the plugins in the build.gradle file as shown below

plugins {
        id 'org.springframework.boot' version '2.3.1.RELEASE'
        id 'io.spring.dependency-management' version '1.0.9.RELEASE'
        id 'java'
        id 'eclipse'
        id 'jacoco'
}

Once the above is added, you can run the following Gradle commands; however, nothing happens yet since we have not yet configured JaCoCo.

./gradlew jacocoTestReport 
./gradlew jacocoTestCoverageVerification

Let’s add the JaCoCo build configs to the build.gradle

jacoco {
    toolVersion = "0.8.5"
    reportsDir = file("$buildDir/jacoco")
}

Let’s create a dependency between the test and jacocoTestReport tasks

1. When the test is run then make execute jacocoTestReport before finishing
2. When the jacocoTestReport is run, then first run the test and after the the jacocoTestReport

test {
    finalizedBy jacocoTestReport
}
jacocoTestReport {
    dependsOn test
}

Here is the complete build.grdle file

Gradle build.gradle with JaCoCo code coverage

Step 4. Generate code coverage report

Now that we have done the basic setup of JaCoCo with Gradle let’s run the coverage report and we can do that using either of the following commands

./gradlew test
./gradlew jacocoTestReport

As shown below when running the ./gradlew test it also runs the jacocoTestReport because of the dependency above

Now navigate to the project in Eclipse, and under build/jacoco/test/html you will find the index.html as shown below

And that’s basically the JaCoCo code coverage report as shown below

We have covered setting up JaCoCo with both Maven and Gradle build tools above.

I have used Eclipse IDE but, its relatively straight forward process and all that matters is setting up the pom.xml and build.gradle

JaCoCo Execution Goals

There are 11 execution goals with JaCoCo, and you can see the list by running mvn jacoco:help and some of them is listed below

1. The command mvn jacoco:check — Checks that the coverage metrics are being met.
2. The command mvn jacoco:dump — Request a dump over TCP/IP from a JaCoCo agent running in TCP server mode.
3. The command mvn jacoco:merge — Merges a set of execution data i.e., jacoco.exec into a single file.
4. The command mvn jacoco:prepare-agent — Prepares a property pointing to the JaCoCo runtime agent that can be passed as a VM argument to the application under test
5. The command mvn jacoco:report — Create a code coverage report in multiple formats (HTML, XML, and CSV).

JaCoCo Rules to Enforce code coverage metrics

You can add rules to your JaCoCo configuration to enforce certain code coverage metrics.

The rules can be applied to the following element types with a list of limits

• BUNDLE
• PACKAGE
• CLASS
• SOURCEFILE
• METHOD

The limits apply to the following counters

• INSTRUCTION
• LINE
• BRANCH
• COMPLEXITY
• METHOD
• CLASS

You can define a minimum or maximum for the following

• TOTALCOUNT
• COVEREDCOUNT
• MISSEDCOUNT
• COVEREDRATIO
• MISSEDRATIO

If nothing is specified, then the default is as follow

rule element: BUNDLE
limit counter: INSTRUCTION
limit value: COVEREDRATIO

Here are some examples from the documentation

This example requires an overall instruction coverage of 80%, and no class must be missed

<rules>
  <rule>
    <element>BUNDLE</element>
    <limits>
      <limit>
        <counter>INSTRUCTION</counter>
        <value>COVEREDRATIO</value>
        <minimum>0.80</minimum>
      </limit>
      <limit>
        <counter>CLASS</counter>
        <value>MISSEDCOUNT</value>
        <maximum>0</maximum>
      </limit>
    </limits>
  </rule>
</rules>

Here is an example where excludes is used. In this example, a line coverage minimum of 50% for every class except test classes is required

<rules>
  <rule>
    <element>CLASS</element>
    <excludes>
      <exclude>*Test</exclude>
    </excludes>
    <limits>
      <limit>
        <counter>LINE</counter>
        <value>COVEREDRATIO</value>
        <minimum>50%</minimum>
      </limit>
    </limits>
  </rule>
</rules>

Here are some Gradle-based examples. As shown below, you can have one or more rules, and each rule can by default apply to BUNDLE or you can narrow it down to CLASS, etc using the element tag.

jacocoTestCoverageVerification {
  violationRules {
    rule {
      element = 'CLASS'      
      limit {
        minimum = 1
      }
      excludes = [
        'package.ClassA',
        'package.ClassB'
      ]    
    }
    rule { 
      limit {
        counter = 'INSTRUCTIONS' 
        value = 'MISSEDRATIO'
        minimum = 0.3 
      } 
    }
  }
}

Let’s distill the above configuration

• We can have more than one rule
• Each rule can be applied to an element type i.e., BUNDLE, PACKAGE, CLASS, SOURCEFILE , and METHOD using the element field.
• The limit takes a counter which can be either INSTRUCTION, LINE, BRANCH, COMPLEXITY, METHOD, or CLASS using the field counter
• The limit takes a value which is either TOTALCOUNT, COVEREDCOUNT, MISSEDCOUNT, COVEREDRATIO, or MISSEDRATIO using the field value
• The limit also takes a minimum or maximum

If no counter is provided for limit then it defaults to INSTRUCTIONS, and if no value is provided then it defaults to COVEREDRATIO.

Conclusion

Thank you for reading this article. I have published an article on how to contribute to open-source if that’s a topic that interests you

It’s never too late to develop habits and know things, and here is an article that I published about things I wish I knew earlier in my career

If you are a Java Programmer and coding standard is something that interests you, then you might find the patterns in the following article useful

Feel free to check out my previous articles https://medium.com/@rhamedy and follow me for future such reads. Please feel free to connect with me on LinkedIn.