Maven Test Throws Exception in Ubuntu

Question

Since an event I do not know, calling mvn test does not work anymore in my Ubuntu 18.04. (running since April). mvn install -DskipTests=true or any other lifecycle phase I tried works well.

If I run for example mvn test in a recent version of Apache Commons IO cloned into /home/reichelt/commons-io/, if shows

[ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /home/reichelt/commons-io && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx25M -jar /home/reichelt/commons-io/target/surefire/surefirebooter6661636703131088733.jar /home/reichelt/commons-io/target/surefire 2018-10-31T18-30-50_926-jvmRun1 surefire4405197448341812663tmp surefire_1048154306090710821475tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:494)
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:441)
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:293)
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
[ERROR]     at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1149)
[ERROR]     at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:978)
[ERROR]     at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:854)
[ERROR]     at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
[ERROR]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
[ERROR]     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
[ERROR]     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
[ERROR]     at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
[ERROR]     at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR]     at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
[ERROR]     at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
[ERROR]     at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
[ERROR]     at org.apache.maven.cli.MavenCli.execute(MavenCli.java:955)
[ERROR]     at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:290)
[ERROR]     at org.apache.maven.cli.MavenCli.main(MavenCli.java:194)
[ERROR]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]     at java.lang.reflect.Method.invoke(Method.java:498)
[ERROR]     at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
[ERROR]     at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
[ERROR]     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
[ERROR]     at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
[ERROR] Caused by: org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /home/reichelt/commons-io && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx25M -jar /home/reichelt/commons-io/target/surefire/surefirebooter6661636703131088733.jar /home/reichelt/commons-io/target/surefire 2018-10-31T18-30-50_926-jvmRun1 surefire4405197448341812663tmp surefire_1048154306090710821475tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:671)
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:533)
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:115)
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:429)
[ERROR]     at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:406)
[ERROR]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ERROR]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[ERROR]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[ERROR]     at java.lang.Thread.run(Thread.java:748)

The target/surefire-reports contains many repetitions of

# Created on 2018-10-31T18:31:48.243
Error: Could not find or load main class org.apache.maven.surefire.booter.ForkedBooter 

Any other project I tried shows similiar behaviour.

I tried downloading maven 3.5.4, unzipping it, adding it to the path by export PATH=/home/reichelt/Downloads/apache-maven-3.5.4-bin/apache-maven-3.5.4/bin:$PATH and running the newer version, but this did not change anything (while mvn -version works correctly and shows the newer version). Same happens for Maven 3.5.0.

Maven error: Could not find or load main class org.codehaus.plexus.classworlds.launcher.Launcher suggests that setting export M3_HOME=/usr/share/maven and export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/ should help, but unfortunately, it did not.

I tried deleting ~/.m2, but unfortunately, this also did not help.

Running this in a docker container with docker run -it ubuntu and running

apt update
apt install -y openjdk-8-jdk maven git
mvn -version
git clone https://github.com/apache/commons-io.git
cd commons-io/
mvn test

yields the same behaviour, so it seems like this is not only an issue of my installation.

Running docker run -it fedora bash and

yum install java-1.8.0-openjdk.x86_64*
yum install wget
wget http://ftp-stud.hs-esslingen.de/pub/Mirrors/ftp.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz
tar -xvf apache-maven-3.5.4-bin.tar.gz 
yum install git
export PATH=/apache-maven-3.5.4/bin:$PATH
mvn -version
git clone https://github.com/apache/commons-io.git
cd commons-io/
mvn test

runs the tests as expected surprisingly.

Has anybody an idea how to solve this issue / work around it?

EDIT Like the message suggests, this seems to be a problem with the ForkedBooter which can not be found. https://maven.apache.org/surefire/maven-surefire-plugin/examples/class-loading.html describes that surefire can be used by an isolated classloader in a booter jar, or by an manifest-only jar. It seems like by default, I am using a manifest only jar which contains something like:

Manifest-Version: 1.0
Class-Path: file:/home/reichelt/.m2/repository/org/apache/maven/surefi
 re/surefire-booter/2.21.0/surefire-booter-2.21.0.jar file:/home/reich
 elt/.m2/repository/org/apache/maven/surefire/surefire-api/2.21.0/sure
 fire-api-2.21.0.jar file:/home/reichelt/.m2/repository/org/apache/mav
 en/surefire/surefire-logger-api/2.21.0/surefire-logger-api-2.21.0.jar
  file:/home/reichelt/commons-io/target/test-classes/ file:/home/reich
 elt/commons-io/target/classes/ file:/home/reichelt/.m2/repository/jun
 it/junit/4.12/junit-4.12.jar file:/home/reichelt/.m2/repository/org/h
 amcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar file:/home/reichelt/.
 m2/repository/org/apache/maven/surefire/surefire-junit4/2.21.0/surefi
 re-junit4-2.21.0.jar
Main-Class: org.apache.maven.surefire.booter.ForkedBooter

This seems fine to me: /home/reichelt/.m2/repository/org/apache/maven/surefire/surefire-booter/2.21.0/surefire-booter-2.21.0.jar is fine and contains a class file for org.apache.maven.surefire.booter.ForkedBooter.

By adding

<useSystemClassLoader>true</useSystemClassLoader>
<useManifestOnlyJar>false</useManifestOnlyJar>

to the pom, builds run again. This seems to be a solution which makes no use of the ForkedBooter. While this is a workaround, I'd still like to understand: Can I further debug this issue, in order to find a reason while this does not work with current Ubuntu? Since it does not work with other maven versions (even in a fresh Ubuntu docker container), but does work on a Fedora container, I assume that the reason somehow is connected to the Ubuntu default configuration.

Answer 1

Funny story: Indeed, this was an OpenJDK update issue.

I read my last history by sudo zcat /var/log/apt/history.log.1.gz | less (depending on logrotating, it was .1, could be anything else). This showed that an update was run this morning:

Start-Date: 2018-10-31  10:38:04
Commandline: /usr/bin/unattended-upgrade
Upgrade: openjdk-8-jdk:amd64 (8u181-b13-0ubuntu0.18.04.1, 8u181-b13-1ubuntu0.18.04.1), openjdk-8-jre:amd64 (8u181-b13-0ubuntu0.18.04.1, 8u181-b13-1ubuntu0.18.04.1), openjdk-8-jdk-headless:amd64 (8u181-b13-0ubuntu0.18.04.1, 8u181-b13-1ubuntu0.18.04.1), openjdk-8-jre-headless:amd64 (8u181-b13-0ubuntu0.18.04.1, 8u181-b13-1ubuntu0.18.04.1)
End-Date: 2018-10-31  10:38:09

So for some reason, OpenJDK 8u181 seems to be incompatible with maven 3.5.2 and surefire. Since I already installed the package upgrade, I downgraded again with sudo apt install openjdk-8-jdk=8u162-b12-1 openjdk-8-jre=8u162-b12-1 openjdk-8-jdk-headless=8u162-b12-1 openjdk-8-jre-headless=8u162-b12-1. Now testing works again.

In https://www.oracle.com/technetwork/java/javase/8u-relnotes-2225394.html, there Java 8u171, 8u172 and 8u181 do not seem to contain updates which make using the maninfest-only jar of surefire impossible, but I only looked shortly above it and did not dive deeper into the bug reports.

EDIT The bug is already reported on OpenJDK: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911925

EDIT 2 This bug is fixed in Java 1.8.0_191 and mvn test runs normal again.

Answer 2

Ran into the same problem today. Not sure of the best long-term fix, but setting the forkCount to 0 for maven-surefire-plugin seems to have fixed it for me for now. Probably makes the tests take longer, unfortunately.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <forkCount>0</forkCount>
    <reuseForks>true</reuseForks>
    <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
  </configuration>
</plugin>

Hope it helps.