Maven: How to run some tests after the "install" phase, during a "release:prepare"?

Question

I have some complex tests that use standalone Maven projects to validate some code. Those Maven projects are programatically packaged by the tests and the resulting .jar files are then used. Those projects use artifacts from my main multi-modules project at the current version, in their pom.xml. In other words, the test Maven projects need to be able to find artifacts provided by my main project.

In an IDE, everything works fine since the current artifacts from my main project are resolved dynamically (no local repository required) when I run the tests. Also, I can install those SNAPSHOT versions before running the tests. But when I want to release a new version of my project, I would need release:prepare to:

• Update the versions (remove all the "-SNAPSHOT").
• Runs all tests that do not end with *.PostInstallTest.java.
• Package the artifacts and install them locally.
• THEN runs the tests that end with *.PostInstallTest.java, since those tests require access to the previously installed artifacts! If the tests fail, no commits would by pushed by release:prepare.

I know this is not perfect since a "bad" version of the artifacts could be installed locally, when the "PostInstallTest" tests fail. But I would prefere that to not having those tests run at all!

Currently, my only working idea is to set an system property when the release profile is used, and have the *.PostInstallTest.java files disable themselves if that property exists. That way, those tests would still work when ran inside my IDE (without the release profile), but wouldn't be executed at all during a release:prepare command. But, again, I would like them to be executed.

I looked at the preparationGoals configuration of the Maven Release Plugin but I'm not sure how it could help me. I also looked at the Maven Failsafe Plugin but it doesn't seem to support the "install" phase.

So, my question: Is there a way to run some tests after the "install" phase, when release:prepare is used (or at the "install" phase, but after the default plugin)?

UPDATE: Here's a quick schema, if it can help understand what's going on:

UPDATE 2: I didn't test Failsafe properly, after all. It does work at the "install" phase! Look at df778899's answer.

Answer 1

Check the following setup. It relies on preparationGoals to execute install. In the configuration of the failsafe plugin, I configured it to run in the install phase.

I believe variations of this approach will work fine as well - you can try unbinding failsafe from any phase (phase set to None), and call it explicitely in preparationGoals (probably this needs some extra config like execution id, but I think you can continue from this point).

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <executions>
                <execution>
                    <phase>install</phase>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-release-plugin</artifactId>
            <version>2.5.3</version>
            <configuration>
                <preparationGoals>clean verify install</preparationGoals>
            </configuration>
            <dependencies>
                <dependency>
                    <!-- Specify the version of maven-scm-plugin to avoid https://issues.apache.org/jira/browse/SCM-682
                    (Maven release fails when releasing from a named branch)
                    -->
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-scm-plugin</artifactId>
                    <version>1.9.5</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

The output (edited a bit to reduce noise) shows that the verify taks launched by preparationGoals are executed after install.

[INFO] --- maven-release-plugin:2.5.3:prepare (default-cli) @ demo ---
...
[INFO] Checking dependencies and plugins for snapshots ...
What is the release version for "demo"? (com.example:demo) 0.0.1: :
What is SCM release tag or label for "demo"? (com.example:demo) demo-0.0.1: :
What is the new development version for "demo"? (com.example:demo) 0.0.2-SNAPSHOT: :
[INFO] Transforming 'demo'...
[INFO] Not generating release POMs
[INFO] Executing goals 'clean verify install'...
[WARNING] Maven will be executed in interactive mode, but no input stream has been configured for this MavenInvoker instance.
[INFO] [INFO] Scanning for projects...
[INFO] [INFO]
[INFO] [INFO] --------------------------< com.example:demo >--------------------------
[INFO] [INFO] Building demo 0.0.1
[INFO] [INFO] --------------------------------[ jar ]---------------------------------
[INFO] [INFO]
[INFO] [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ demo ---
[INFO] [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ demo ---
[INFO] [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ demo ---
[INFO] [INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ demo ---
[INFO] [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ demo ---
[INFO] [INFO] --- maven-surefire-plugin:2.22.1:test (default-test) @ demo ---
[INFO] [INFO] --- maven-jar-plugin:3.1.1:jar (default-jar) @ demo ---
[INFO] [INFO] --- spring-boot-maven-plugin:2.1.4.RELEASE:repackage (repackage) @ demo ---
[INFO] [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ demo ---
[INFO] [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ demo ---
[INFO] [INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ demo ---
[INFO] [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ demo ---
[INFO] [INFO] --- maven-surefire-plugin:2.22.1:test (default-test) @ demo ---
[INFO] [INFO] --- maven-jar-plugin:3.1.1:jar (default-jar) @ demo ---
[INFO] [INFO] Building jar: demo-0.0.1.jar

[INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ demo ---
[INFO] [INFO] Installing demo-0.0.1.jar to 

[INFO] [INFO] --- maven-failsafe-plugin:2.22.1:integration-test (default) @ demo ---
[INFO] [INFO]
[INFO] [INFO] --- maven-failsafe-plugin:2.22.1:verify (default) @ demo ---
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] BUILD SUCCESS
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] Total time: 13.784 s

[INFO] Checking in modified POMs...

Answer 2

I don't think this is an answer as yet, but to expand on @Lesiak's point, the Failsafe plugin looks fine by itself in the install phase. For example:

<plugin>
    <artifactId>maven-failsafe-plugin</artifactId>
    <executions>
        <execution>
            <phase>install</phase>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Results in this output:

...
[INFO] --- maven-install-plugin:2.4:install (default-install) @ it-test ---
[INFO] Installing ...\target\it-test-0.0.1-SNAPSHOT.jar to ...\.m2\repository\group\it-test\0.0.1-SNAPSHOT\it-test-0.0.1-SNAPSHOT.jar
[INFO] Installing ...\pom.xml to ...\.m2\repository\group\it-test\0.0.1-SNAPSHOT\it-test-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M3:integration-test (default) @ it-test ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TheIT
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.093 s - in TheIT
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M3:verify (default) @ it-test ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

Note how the maven-failsafe-plugin:3.0.0-M3:integration-test and maven-failsafe-plugin:3.0.0-M3:verify goals are run at the end - in the install phase.