getting code coverage for java code with scala tests

Question

The project is a multi module maven project with 90% of the source code written in Java (there's a small bit in Scala). The unit tests are 80/20 java/scala, and the integration tests are 20/80 java/scala.

I tried Clover, but (at this time) it doesn't support scala.

I tried Jacoco. First I ran into problems getting any results due to the mutli-module configuration, but now using Sonar I've got the java coverage shown (thanks http://www.aheritier.net/maven-failsafe-sonar-and-jacoco-are-in-a-boat/). I used timezra (http://timezra.blogspot.com/2013/10/jacoco-and-scala.html) with jacoco, but that only analyzed the small bit of source code that is scala.

I started to try Scoverage, but that seems to have the same problem as timezra (it analyzes scala-to-scala, not the mix that I have). I therefor haven't even tried scct or undercover.

---

Is there any tool that handles mixed java/scala?

Answer 1

It is possible to see the mix of the two languages coverage. With sonar is not possible, because it only analyzes one kind of language.

If you use jenkins, and configure a step that collect coverage reports(Jacoco), you will see the mixing the two languages.

Answer 2

to answer my own question, Jacoco does analyze coverage for Scala tests and Java code. I haven't seen anything that does the opposite (but we don't have any Java tests for Scala code, so that didn't matter).

As for combining coverage, Rado's answer of using two tools makes sense.

---

Here's the changes I did to my pom to get coverage. NOTE, I'm creating Jacoco reports when the test runs, then later using Sonar for more analysis:

properties:

    <!-- Jacoco and Sonar config properties   -->
    <sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
    <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
    <jacoco.version>0.7.2.201409121644</jacoco.version>
    <sonar-jacoco-listeners.version>1.4</sonar-jacoco-listeners.version>

    <jacoco.outputDir>${basedir}/target/</jacoco.outputDir>
    <jacoco.out.ut.file>jacoco-ut.exec</jacoco.out.ut.file>
    <jacoco.out.it.file>jacoco-it.exec</jacoco.out.it.file>

    <sonar.jacoco.reportPath>${jacoco.outputDir}/${jacoco.out.ut.file}</sonar.jacoco.reportPath>
    <sonar.jacoco.itReportPath>${jacoco.outputDir}/${jacoco.out.it.file}</sonar.jacoco.itReportPath>

Failsafe:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.17</version>
        <configuration>
          <argLine>-Xms512m -Xmx1024m ${jacoco.agent.it.arg}</argLine>
          <properties>
            <property>
              <name>listener</name>
              <value>org.sonar.java.jacoco.JUnitListener</value>
            </property>
          </properties>
          <reportsDirectory>${jacoco.outputDir}/surefire-reports</reportsDirectory>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>         

jacoco:

      <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>${jacoco.version}</version>
        <executions>
          <execution>
            <id>prepare-ut-agent</id>
            <phase>process-test-classes</phase>
            <goals>
              <goal>prepare-agent</goal>
            </goals>
            <configuration>
              <destFile>${sonar.jacoco.reportPath}</destFile>
              <propertyName>jacoco.agent.ut.arg</propertyName>
              <append>true</append>
            </configuration>
          </execution>
          <execution>
            <id>prepare-it-agent</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>prepare-agent-integration</goal>
            </goals>
            <configuration>
              <destFile>${sonar.jacoco.itReportPath}</destFile>
              <propertyName>jacoco.agent.it.arg</propertyName>
              <append>true</append>
            </configuration>
          </execution>
          <execution>
            <id>default-report</id>
            <phase>package</phase>
            <goals>
              <goal>report</goal>
            </goals>
            <configuration>
              <dataFile>${sonar.jacoco.reportPath}</dataFile>
            </configuration>
          </execution>
          <execution>
            <id>integration-report</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>report-integration</goal>
            </goals>
            <configuration>
              <dataFile>${sonar.jacoco.itReportPath}</dataFile>
            </configuration>
          </execution>
        </executions>
      </plugin>

surefire:

        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.17</version>
          <configuration>
            <argLine>${jacoco.agent.ut.arg} -Xms512m -Xmx1024m</argLine>
            <skipTests>false</skipTests>
            <properties>
              <property>
                <name>listener</name>
                <value>org.sonar.java.jacoco.JUnitListener</value>
              </property>
            </properties>
            <reportsDirectory>${jacoco.outputDir}/surefire-reports</reportsDirectory>
          </configuration>
        </plugin>

added dependencies:

    <dependency>
      <groupId>org.codehaus.sonar-plugins.java</groupId>
      <artifactId>sonar-jacoco-listeners</artifactId>
      <version>${sonar-jacoco-listeners.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.codehaus.sonar.plugins</groupId>
      <artifactId>sonar-surefire-plugin</artifactId>
      <version>3.3.2</version>
      <scope>test</scope>
    </dependency>

Answer 3

I would suggest to have two separate coverage tools for Java and Scala. More specifically, use Scoverage for Scala (with plugin for Sonar).

The reason is that for Java you would probably like to measure line coverage where for Scala it's much better to measure statement coverage. Simply said because there are many statements on a single line in Scala and you would like to measure which of them were invoked. I've written an article about this.