CODEWITHSUNDEEP
javamavenjunitmaven-surefire-plugin
user118967
user118967

Reputation: 5722

Surefire Maven plugin: "Corrupted STDOUT by directly writing to native stream in forked JVM"

My JUnit tests are failing when running them through Maven and the Surefire plugin (version information below). I see the error message:

Corrupted STDOUT by directly writing to native stream in forked JVM 4. See FAQ web page and the dump file C:\(...)\target\surefire-reports\2019-03-20T18-57-17_082-jvmRun4.dumpstream

The FAQ page points out some possible reasons but I don't see how I can use this information to start solving this problem:

Corrupted STDOUT by directly writing to native stream in forked JVM

If your tests use native library which prints to STDOUT this warning message appears because the library corrupted the channel used by the plugin in order to transmit events with test status back to Maven process. It would be even worse if you override the Java stream by System.setOut because the stream is also supposed to be corrupted but the Maven will never see the tests finished and build may hang.

This warning message appears if you use FileDescriptor.out or JVM prints GC summary.

In that case the warning is printed "Corrupted STDOUT by directly writing to native stream in forked JVM", and a dump file can be found in Reports directory.

If debug level is enabled then messages of corrupted stream appear in the console.

It refers to some native library printing out to STDOUT directly but how can I figure out which one, and even if I do, how do I deal with this issue if I need the library for my project?

It mentions "debug level" but it is unclear if this means Maven's debug level or Surefire plugin's debug level. I enabled Maven's debug but I don't see the console outputs as mentioned by the FAQ. And Surefire's debug option seems to be about pausing tests and waiting for a debugger to be connected to the process, not simply showing more information on the console.

The dump files also don't seem very helpful:

# Created on 2019-03-20T18:42:58.323
Corrupted STDOUT by directly writing to native stream in forked JVM 2. Stream 'FATAL ERROR in native method: processing of -javaagent failed'.
java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'FATAL ERROR in native method: processing of -javaagent failed'.
    at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:511)
    at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:209)
    at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:176)
    at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:88)
    at java.base/java.lang.Thread.run(Thread.java:834)

So, how can I solve this problem?

Update: requested configuration information below.

I'm using OpenJDK 11 (Zulu distribution) on Windows 10, Maven 3.5.3, and Surefire 2.21.0 (full configuration below).

I'm running Maven from Eclipse using the "Run As..." context menu option on the pom.xml file, but obtain the same results when running it on the console.

I had never heard of JaCoco before the first comment to this question, but I see several error messages mentioning it:

[ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was cmd.exe /X /C ""C:\Program Files\Zulu\zulu-11\bin\java" -javaagent:C:\\Users\\E26638\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.0\\org.jacoco.agent-0.8.0-runtime.jar=destfile=C:\\Users\\E26638\\git\\aic-expresso\\target\\jacoco.exec -Xms256m -Xmx1028m -jar C:\Users\E26638\AppData\Local\Temp\surefire10089630030045878403\surefirebooter8801585361488929382.jar C:\Users\E26638\AppData\Local\Temp\surefire10089630030045878403 2019-03-21T21-26-04_829-jvmRun12 surefire10858509118810158083tmp surefire_115439010304069944813tmp"
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1

This is the Surefire Maven plugin configuration:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.21.0</version>
        <configuration>
            <skipTests>${skipUnitTests}</skipTests>
            <testFailureIgnore>false</testFailureIgnore>
            <forkCount>1.5C</forkCount>
            <reuseForks>true</reuseForks>
            <parallel>methods</parallel>
            <threadCount>4</threadCount>
            <perCoreThreadCount>true</perCoreThreadCount>
            <reportFormat>plain</reportFormat>
            <trimStackTrace>false</trimStackTrace>
            <redirectTestOutputToFile>true</redirectTestOutputToFile>
        </configuration>
    </plugin>

Upvotes: 107

Views: 98004

Answers (14)

Arun Ramakrishnan
Arun Ramakrishnan

Reputation: 128

I have version 3.1.2 of surefire and randomly ran into this issue. I do need to restart my laptop and try again. It's a macbook M1, I don't know if that might be adding to the issue.

However, adding <argLine>-Xmx1024m</argLine> to the plugin configuration in the Maven POM file did solve it for me.

Upvotes: 0

Manuel
Manuel

Reputation: 4228

We faced the same problem (OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (11.0.17+8, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)). We already had a jacoco-plugin version of 0.8.5. A downgrade to 0.8.4 did not help. We dug deepter into the dump file and it (i.e., the gitlab runner) spit out:

Corrupted STDOUT by directly writing to native stream in forked JVM 1. Stream '# /builds/mygroup/myproject/hs_err_pid114.log'.

In that file we discovered

Internal Error (sharedRuntime.cpp:1262), pid=114, tid=115 guarantee((retry_count++ < 100)) failed: Could not resolve to latest version of redefined method

... which lead to this bug issue: https://bugs.openjdk.org/browse/JDK-6776659

The issue should be fixed in Java 16. Upgrading at this point was not possible. Luckily, the thread creator provided a workaround:

CUSTOMER SUBMITTED WORKAROUND : -Xint or -server

As we were running a maven docker image in the gitlab runner, we had to set the forkCount to 0 in the maven surefire plugin configuration:

<configuration>
  <forkCount>0</forkCount>
</configuration>

That solved the VM crash. Unfortunately, we also had a testcoverage check (JaCoCo) and that was no longer working, as the surefire plugin created a separate VM for e.g., test classes instrumentatlization.

target/site/jacoco/jacoco.csv: No such file or directory

Finally, we had to use the argLine configuration of surefire (2, 3)

<configuration>
  <argLine>-Xint ${argLine}</argLine>
</configuration>

.. and it worked.

Upvotes: 3

Benjamin Peter
Benjamin Peter

Reputation: 4038

We are using log4j backend and could also fix it using follow set to true.

<Appenders>
    <Console name="Console" target="SYSTEM_OUT">
        <PatternLayout pattern="${messagePattern}" />
        <follow>true</follow>
    </Console>
</Appenders>

(We had the same error message but without jacoco being involved. But I can confirm that setting the forkNode in maven-surefire-plugin to TCP did also work.)

Upvotes: 5

sjngm
sjngm

Reputation: 12861

In my case I moved my development to a new PC and didn't have all our company library-dependencies in my Maven-repo yet. So when Maven ran there was such a library missing and I had to install it with mvn install:install-file ....

Therefore, it's important to read the latest surefire-logs as it suggests those things.

No idea, why the surefire-plugin doesn't just print that conflicting line on STDOUT to the console, so it would be obvious in less than a second.

Upvotes: 0

miroana
miroana

Reputation: 552

This issue happens to me too at random

I'm using

IntelliJ IDEA 2020 (Community Edition)
Surefire plugin (3.0.0-M5)
Maven 3.3.9
AdoptOpenJDK 11

And when it happens usually Windows 10 after several minutes shows a beautiful blue screen of death.

And then after a restart everything goes back to normal

Upvotes: 0

Matthew Read
Matthew Read

Reputation: 1859

If you are unable to upgrade to the latest JaCoCo version, I was also able to fix this for my project by setting forkCount to 0:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.22.0</version>
  <configuration>
    <forkCount>0</forkCount>
  </configuration>
</plugin>

Upvotes: 12

Marco
Marco

Reputation: 41

None of the listed answers helped in our case. The issue began after we've upgraded from Java 8 to Java 11.

Note: updating Surefire plugin was not possible in our case since it has broken some mechanisms the tests in our projects rely on - inspecting the issue took too long so we started identifying the root cause in Jacoco for that behaviour.

After some debugging and JVM dumps we found the cause: in our case we had JavaFX dependencies on the classpath which were loaded automatically by a resolver util. Loading these classes with jacoco enabled led to the JVM crash (without jacoco - encapsulated in a profile "coverage" in our case - did run fine). Excluding the loading of classes from JavaFX libraries (were not needed in our case) fixed the issue. Tests are running fine now without JVM crashs.

The exact class that led to the JVM crash (or at least the last that were loaded before) was in our case: com.sun.javafx.logging.jfr.JFRPulsePhaseEvent Jar: javafx-base-12-win.jar

Hint: in many IDEs you can debug the Maven build with specific profile and check what is going on exactly.

Using Jacoco 0.8.6 and Surefire plugin 2.22.2

Upvotes: 2

seinecle
seinecle

Reputation: 10778

For me it was upgrading org.testng to the latest version (7.3.0)

Upvotes: 0

havryliuk
havryliuk

Reputation: 311

I was getting this error when running Maven build in Intelij Idea. I had a couple of projects open in separate windows and had other strange errors in a different project.

Solved for me by closing all the Intellij Idea windows and re-opening the project. No dependencies versions were changed.

Upvotes: -1

Patrick
Patrick

Reputation: 1567

I was running into this issue when running my Junit tests using a custom Runner. If I made any output to System.out or System.err in my custom runner or in my test class, this exact warning would show up. In my case the problem was not caused by some older Jacoco version. Updating the surefire plugin to version 2.22.2 or the more recent 3.0.0-M4 did not solve the issue.

According to the Jira issue SUREFIRE-1614, the problem will be fixed in the 3.0.0-M5 release of the maven-surefire-plugin (not released as of May 21st 2020).

Update The Maven Surefire plugin version 3.0.0-M5 has now been released. In your pom.xml you can do the following:

    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>3.0.0-M5</version>
      <configuration>
        <!-- Activate the use of TCP to transmit events to the plugin -->
        <forkNode implementation="org.apache.maven.plugin.surefire.extensions.SurefireForkNodeFactory"/>
      </configuration>
    </plugin>

Original answer

If you cannot wait for the release of the 3.0.0-M5 plugin, you can use the "SNAPSHOT" version of the plugin. It did fix the issue for me. You have to enable some specific setting in the plugin so that the plugin uses TCP instead of the standard output/error to obtain the events raised in your tests. Configuration changes below:

In my pom.xml:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

...
  <!-- Add the repository to download the "SNAPSHOT" of maven-surefire-plugin -->
  <pluginRepositories>
    <pluginRepository>
      <id>apache.snapshots</id>
      <url>https://repository.apache.org/snapshots/</url>
    </pluginRepository>
  </pluginRepositories>

  <build>
    <pluginManagement>
      <plugins>
    ...
    <artifactId>maven-surefire-plugin</artifactId>
      <!-- Use the SNAPSHOT version -->
      <version>3.0.0-SNAPSHOT</version>
      <configuration>
        <!-- Activate the use of TCP to transmit events to the plugin -->
        <forkNode implementation="org.apache.maven.plugin.surefire.extensions.SurefireForkNodeFactory"/>
      </configuration>
    </plugin>

Upvotes: 39

Werner Keil
Werner Keil

Reputation: 642

The newer Surefire plugin versions are completely buggy and broken. for me (tested all the way up to Java 12) the only solution was to stick with 2.20.

Don't use 2.20.1 either, that failed with a NPE, although maybe it is specific to particular tests, but I don't have time to investigate that.

Upvotes: -3

jhas
jhas

Reputation: 597

What solved it for me is upgrading maven surefire plugin to 2.22.2

Upvotes: 0

Petar Tahchiev
Petar Tahchiev

Reputation: 4386

For me it was updating the failsafe plugin from 2.22.0 to 2.22.2

Upvotes: 22

Nemesis
Nemesis

Reputation: 2918

Run in the same problem while migrating project from JAVA 8 to JAVA 11, upgrading jacoco-plugin from 0.8.1 to 0.8.4 did the job.

Analysing maven dependencies, seeing from where jacoco is pulled and then fixing the version should solve the issue.

Upvotes: 126

Related Questions