How to check minimum code coverage for multi-module maven project with jacoco?

Question

I want to use jacoco maven plugin for checking minimum level of code coverage during build process using 'check' goal.

For one-module project everything works fine. But for multi-module I want to check average level of code coverage from all modules, but check goal checks every module separately.

For example, module1 has 70% of code coverage, module2 has 100% code coverage, in average for all lines from both modules code coverage is 85%. And I am trying to set code coverage for all project to 80%, but it fails because of first module.

From pom:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.6.201602180812</version>
    <executions>
        <execution>
            <id>default-prepare-agent</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>default-report</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
        <execution>
            <id>default-check</id>
            <goals>
                <goal>check</goal>
            </goals>
            <configuration>
                <rules>
                    <rule>
                        <element>BUNDLE</element>
                        <limits>
                            <limit>
                                <counter>COMPLEXITY</counter>
                                <value>COVEREDRATIO</value>
                                <minimum>0.80</minimum>
                            </limit>
                        </limits>
                    </rule>
                </rules>
            </configuration>
        </execution>
    </executions>
</plugin>

Answer 1

Not possible (as of now, and hopefully in future too). But I have an alternate way of doing this.

Jacoco has a goal to create an aggregated report for multi module projects, but it doesn't create a jacoco.exec file which is required for checking the coverage metrics.

One way is to create the aggregated report using report-aggregate goal, and separately generate jacoco.exec files for each module and then combine it using jacoco-merge goal.

However this method is not that great as it only cover the direct code coverage. e.g. -> Suppose your module A has some tests which covers some lines of code in module B, in this case merge will not consider this module level differences, it is because you are first generating module level exec file and then merging it into one.

I found a better approach for this check at an aggregate level. This doesn't require any exec files, you can do it using the report-aggregate goal only!! If you have noticed the report-aggregate goal do creates html, xml and csv files for the aggregated report, we can simple scrape data from these files and write our validations!

Here is what I have implemented using XML file

        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>${jacoco-maven-plugin.version}</version>
            <executions>
                <execution>
                    <id>prepare-agent</id>
                    <goals>
                        <goal>prepare-agent</goal>
                    </goals>
                </execution>
                <!--  Aggregate the report after tests run -->
                <execution>
                    <id>report-aggregate</id>
                    <phase>verify</phase>
                    <goals>
                        <goal>report-aggregate</goal>
                    </goals>
                    <configuration>
                        <outputDirectory>
                            ${project.basedir}/target/jacoco-reports
                        </outputDirectory>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.6.0</version>
            <executions>
                <execution>
                    <phase>verify</phase>
                    <goals>
                        <goal>exec</goal>
                    </goals>
                    <configuration>
                        <executable>python3</executable>
                        <arguments>
                            <argument>check_coverage.py</argument>
                            <argument>${code-coverage-percentage}</argument>
                        </arguments>
                    </configuration>
                </execution>
            </executions>
        </plugin>

check_coverage.py

import xml.etree.ElementTree as ET
import sys

# Path to the JaCoCo XML report file
jacoco_report_file = 'target/jacoco-reports/jacoco.xml'

# Get the required coverage from the command-line argument
try:
    required_coverage = float(sys.argv[1])  # Pass as first argument
except IndexError:
    print("Usage: python check_coverage.py <required_coverage>")
    sys.exit(1)
except ValueError:
    print("Invalid value for required coverage, please provide a numeric value.")
    sys.exit(1)


try:
    # Parse the XML file
    tree = ET.parse(jacoco_report_file)
    root = tree.getroot()

    # Find the line coverage data
    counter = root.findall(".//counter[@type='LINE']")
    if not counter:
        print("Could not find line coverage data in the JaCoCo report.")
        exit(1)

    # Pick the last occurrence
    last_counter = counter[-1]

    # Extract the 'missed' and 'covered' values from the XML
    missed = int(last_counter.get('missed', 0))
    covered = int(last_counter.get('covered', 0))
    print(missed)
    print(covered)

    # Calculate the total lines and the line coverage percentage
    total_lines = missed + covered
    if total_lines == 0:
        print("No lines were executed or missed, cannot calculate coverage.")
        exit(1)

    line_coverage = (covered / total_lines) * 100

    # Output the calculated coverage
    print(f"Line Coverage: {line_coverage:.2f}%")

    # Check if the coverage meets the required threshold (e.g., 80%)
    # required_coverage = 80.0  # Set your desired threshold here
    if line_coverage < required_coverage:
        print(f"Coverage is below the required threshold of {required_coverage}%.")
        exit(1)
    else:
        print("Coverage meets the required threshold.")
        exit(0)

except FileNotFoundError:
    print(f"JaCoCo aggregate report not found at {jacoco_report_file}.")
    exit(1)

except ET.ParseError as e:
    print(f"Error parsing the JaCoCo XML report: {e}")
    exit(1)

Simply adjust code-coverage-percentage according to your requirement.

Answer 2

Source: https://www.jacoco.org/jacoco/trunk/doc/check-mojo.html

Use Bundle with counter Intruction, this will check overall code coverage for the whole project:

<rules>
<rule>
    <element>BUNDLE</element>
    <limits>
        <limit>
            <counter>INSTRUCTION</counter>
            <value>COVEREDRATIO</value>
            <minimum>0.80</minimum>
        </limit>
    </limits>
</rule>

Answer 3

Short answer: not possible (at the time of writing) using merely Maven and Jacoco.

from official jacoco github page:

The current JaCoCo Maven goals work on single modules only: Tests are executed within the module and contribute coverage only to code within the same module. Coverage reports are created for each module separately. There is no built-in support for cross-module coverage or combined reports for multiple modules.

Hence your requirements cannot be met merely using Maven and Jacoco. You can however use a common approach in enterprise settings: Sonarqube, which will process the generated jacoco files (i.e. jacoco.exec) and aggregate reporting and governance via its Jacoco integration (provided out-of-the-box on its latest versions).