Test suite hangs when run with maven. Runs fine with a Junit run configuration in IDEA

Question

I've simplified our project down to a few files to try and figure out an issue where maven hangs mid suite. the "Junit run configuration" I mentioned in the title, is just the default configuration that gets created when you run SmokeTestSuite.javain intellij IDEA.

SmokeTestSuite.java

package com.parallelTest;

import org.junit.runner.RunWith;
import org.junit.runners.Suite;

@RunWith(Suite.class)
@Suite.SuiteClasses({
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
        Test001.class,
})
public class SmokeTestSuite {}

I duplicated the Test001 line to simulate having more tests. I noticed that 256 tests (each Test001 parameterizes into 8 test cases) would complete sometimes when running with maven, but 512 tests would always hang, and likewise 128 tests would never hang.

Test001.java

package com.parallelTest;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;

import java.util.ArrayList;
import java.util.List;

@RunWith(Parameterized.class)
public class Test001  {
    private static final Logger logger = LogManager.getLogger(Test001.class);
    private static final String[] DRIVER_TYPES;
    private static final String[] LANGUAGES;
    private static final String[] REGIONS;
    private static String userSubmittedServer = System.getProperty("server");

    static {
        {
            DRIVER_TYPES = new String[] {
                    Constants.BrowserTypes.CHROME,
//                    Constants.BrowserTypes.FIREFOX,
            };
        } // DRIVER_TYPES
        {
            LANGUAGES = new String[] {
                    Constants.Languages.ENGLISH,
//                    Constants.Languages.FRENCH,
            };
        } // LANGUAGES
        {
            REGIONS = new String[] {
                    Constants.Regions.AA,
                    Constants.Regions.BB,
                    Constants.Regions.CC,
                    Constants.Regions.DD,
                    Constants.Regions.EE,
                    Constants.Regions.FF,
                    Constants.Regions.GG,
                    Constants.Regions.HH,
            };
        } // REGIONS

        if (userSubmittedServer == null) {
            userSubmittedServer = Constants.DEFAULT_SERVER;
        }
        else {
            userSubmittedServer = userSubmittedServer.toLowerCase();
        }
    }

    public Test001(String browserName, String region, String language) {
        logger.info(browserName);
        logger.info(region);
        logger.info(language);
        String server = userSubmittedServer;
        logger.info(server);
    }

    @Parameterized.Parameters(name = "{0}|{1}|{2}")
    public static List<String[]> parameterSetup() {
        List<String[]> variants = new ArrayList<>();

        for (String browserName : DRIVER_TYPES) {
            for (String region : REGIONS) {
                for (String language : LANGUAGES) {
                    String[] tempStringList = {browserName, region, language};
                    variants.add(tempStringList);
                }
            }
        }

        return variants;
    }

    @Test
    public void testMethod01() {
        logger.info("Test001.testMethod01()");
    }
}

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <groupId>com.parallelTest</groupId>
    <artifactId>parallel-test</artifactId>
    <version>1.0-SNAPSHOT</version>
    <modelVersion>4.0.0</modelVersion>
    <packaging>jar</packaging>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>

                <configuration>
                    <source>8</source>
                    <target>8</target>
                    <executable>C:\Program Files\Java\jdk-12.0.1\bin\javac.exe</executable>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <configuration>
                    <parallel>classes</parallel>
                    <useUnlimitedThreads>false</useUnlimitedThreads>
                    <perCoreThreadCount>true</perCoreThreadCount>
                    <forkCount>1</forkCount>
                    <threadCount>2</threadCount>
                    <rerunFailingTestsCount>0</rerunFailingTestsCount>
                    <parallelTestsTimeoutForcedInSeconds>7200</parallelTestsTimeoutForcedInSeconds>
                    <useSystemClassLoader>false</useSystemClassLoader>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.13.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.13.0</version>
        </dependency>
    </dependencies>
</project>

The constants file is just a bunch of static final strings, it doesn't really matter what their values are. I can see tests printing to the app.log when running with maven, but the main process never ends. Any help with this would be awesome!

EDIT: I should mention these tests are normally selenium tests and each test spins up a chromedriver.exe. So running with infinite threads isn't an option.

Answer 1

It looks like the issue lies somewhere in the threadCount setting. If I replace threadCount with threadCountClasses everything works as expected. Based on the surefire docs, I thought my previous settings would have also worked. ::shrug::