Maven Surefire Plugin version 3.0.0-M5 ignored include test

Question

I have configured a surefire plugin in pom.xml to run a Testsuite. With version 3.0.0-M3 works all fine, but if I switch to version 3.0.0-M5 test will igore and run 0 tests.

What can I do? What I am doing wrong?

Maven Log with version M3:

16:35:00  [INFO] --- maven-surefire-plugin:3.0.0-M3:test (default-test) @ bitone-appium ---
16:35:01  [INFO] 
16:35:01  [INFO] -------------------------------------------------------
16:35:01  [INFO]  T E S T S
16:35:01  [INFO] -------------------------------------------------------
16:35:01  [INFO] Running Android All Tests Suite
.
.
.
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.921 s - in Android All Tests Suite
16:36:19  [INFO] 
16:36:19  [INFO] Results:
16:36:19  [INFO] 
16:36:19  [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
16:36:19  [INFO] 
16:36:19  [INFO] 

Maven Log with version M5:

[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ xxxx-appium ---
[INFO] Building jar: /Users/xxxx/Projects/xxxx-testautomatisierung/xxxx-appium/target/xxxx-appium-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ xxxx-appium ---
[INFO] Installing ...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.563 s
[INFO] Finished at: 2022-01-10T15:26:33+01:00
[INFO] ------------------------------------------------------------------------

Process finished with exit code 0

part of pom.xml:

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <java.version>8</java.version>
    <maven.compiler.plugin.version>3.8.1</maven.compiler.plugin.version>
    <maven-clean.version>2.5</maven-clean.version>
    <maven.surefire.plugin.version>3.0.0-M3</maven.surefire.plugin.version>
    <maven-javadoc.version>3.2.0</maven-javadoc.version>
    <junit5.jupiter.version>5.8.1</junit5.jupiter.version>
    <junit5.jupiter.suite.version>1.7.0</junit5.jupiter.suite.version>
    <junit5.junit.platform.launcher>1.8.1</junit5.junit.platform.launcher>
    <junit5.junit.platform-runner>1.8.1</junit5.junit.platform-runner>
    <junit5.junit.platform-commons>1.8.1</junit5.junit.platform-commons>
    <junit5.junit.suite-engine>1.8.1</junit5.junit.suite-engine>
    <junit5.junit.platform-params>5.8.1</junit5.junit.platform-params>
    <appium-java-client.version>7.5.1</appium-java-client.version>
    <selenium-java.version>3.141.59</selenium-java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>${junit5.jupiter.version}</version>
        <!--<scope>test</scope>-->
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-launcher</artifactId>
        <version>${junit5.junit.platform.launcher}</version>
        <!--<scope>test</scope>-->
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-runner</artifactId>
        <version>${junit5.junit.platform-runner}</version>
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-commons</artifactId>
        <version>${junit5.junit.platform-commons}</version>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-params</artifactId>
        <version>${junit5.junit.platform-params}</version>
        <!--<scope>test</scope>-->
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-suite-engine</artifactId>
        <version>${junit5.junit.suite-engine}</version>
    </dependency>
    <dependency>
        <groupId>io.appium</groupId>
        <artifactId>java-client</artifactId>
        <version>${appium-java-client.version}</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>${selenium-java.version}</version>
    </dependency>
</dependencies>
<build>
    <testResources>
        <testResource>
            <directory>src/test/resources</directory>
            <filtering>true</filtering>
        </testResource>
    </testResources>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>${maven.compiler.plugin.version}</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <includes>
                    <include>${testsuite}.java</include>
                </includes>
            </configuration>
        </plugin>
    </plugins>
</build>

Maven command:

mvn clean install -Dtestsuite=AndroidAllTestsSuite

AndroidAllTestSuite.java:

import android.login.LoginTest;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;
import org.junit.platform.suite.api.SuiteDisplayName;

@Suite
@SelectClasses({
        LoginTest.class
})
@SuiteDisplayName("Android All Tests Suite")
public class AndroidAllTestsSuite {
}

LoginTest.java:

import org.junit.jupiter.api.*;

public class LoginTest {
    @Test
    public void loginTest() {
        Assertions.assertTrue(true);
    }
}

Answer 1

I'm using TestNG and met similar issue with 3.0.0-M5. It seems surefire begin to prefer Junit5 over TestNG since this version.

see https://github.com/apache/maven-surefire/commit/d5bbb3f9ae37ff5ad2bb7e8b3c11d1f24ea6f041#diff-38e379eb63d7dcd2deb45902b4a494517e4ce54c29913bcf74e7d9243ebb7011R1175 AbstractSurefireMojo.java line 1175.

I solved it by explicit add org.apache.maven.surefire:surefire-testng to surefire plugin dependency. ref: https://stackoverflow.com/a/55282597/1676726

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <dependencies>
                <dependency>
                    <groupId>org.apache.maven.surefire</groupId>
                    <artifactId>surefire-testng</artifactId>
                    <version>${maven.surefire.version}</version>
                </dependency>
            </dependencies>
        </plugin>

Answer 2

Thank you, I could solve the problem. I had been used @BeforeClass (JUnit 4) and Jupiter (JUnit5). That was my problem.

Answer 3

Try to declare surefire-plugin in this way:

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M5</version>
    <dependencies>
      <dependency>
        <groupId>org.apache.maven.surefire</groupId>
        <artifactId>surefire-junit-platform</artifactId>
        <version>3.0.0-M5</version>
      </dependency>
    </dependencies>
  </plugin>

or this:

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M5</version>
    <dependencies>
      <dependency>
        <groupId>org.apache.maven.surefire</groupId>
        <artifactId>surefire-junit47</artifactId>
        <version>3.0.0-M5</version>
      </dependency>
    </dependencies>
  </plugin>

---

Some references

https://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html https://mvnrepository.com/artifact/org.apache.maven.surefire/surefire-junit47 https://mvnrepository.com/artifact/org.apache.maven.surefire/surefire-junit-platform