CODEWITHSUNDEEP
mavenmaven-pluginmaven-failsafe-plugin
Jack
Jack

Reputation: 491

maven-failsafe-plugin Failures and BUILD SUCCESS?

I want to use maven-failsafe-plugin to run some integration tests. If any test fails, I want Maven to fail the build and not BUILD SUCCESS.

Tests run: 103, Failures: 1, Errors: 0, Skipped: 26
[INFO] BUILD SUCCESS*


how can I configure it, that build not success is?

My failsafe plugin is configured as:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>${failsafe.version}</version>
    <configuration>
        <systemProperties>
            <CI_INTEGRATION_OVERRIDE_PATH>${basedir}/..</CI_INTEGRATION_OVERRIDE_PATH>
        </systemProperties>
        <includes>
            <include>**/integration/**/*.java</include>
        </includes>
        <excludes>
            <exclude>**/integration/**/*TestSuite.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Upvotes: 23

Views: 16120

Answers (4)

mihca
mihca

Reputation: 1257

I had a similar issue with Jenkins builds.

  • Running the tests locally resulted in failed tests and thus failed build
  • Running the tests on Jenkins resulted in failed tests but "BUILD SUCCESS"

Solution: in job configuration set MAVEN_OPTS to -Dmaven.test.failure.ignore=false.

Jenkins by default seems to ignore failed integration tests.

See also https://stackoverflow.com/a/28684048/4412885

Upvotes: 0

Ben Mathews
Ben Mathews

Reputation: 2998

As Andrew pointed out, the correct solution is to use failsafe as intended. The integration-test goal is specifically designed not to fail the build. If you want to fail the build, call mvn verify or mvn failsafe:verify

For the verify goal to work, it needs to read the results of the integration test which by default are written to ${project.build.directory}/failsafe-reports/failsafe-summary.xml so make sure that is getting generated.

Also, you have to make sure you bind your maven-failsafe-plugin configuration to both the integration-test goal and the verify goal in the executions part.

Failure to add either of those will lead to maven succeeding the build instead of failing it when integration tests fail.

Upvotes: 21

Jack
Jack

Reputation: 491

solution.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <skip>true</skip>
  </configuration>
  <executions>
    <execution>
      <id>unit-test</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <skip>false</skip>
        <excludes>
          <exclude>**/*IntegrationTest.java</exclude>
        </excludes>
      </configuration>
      </execution>
      <execution>
        <id>integration-test</id>
        <phase>integration-test</phase>
        <goals>
          <goal>test</goal>
        </goals>
        <configuration>
          <skip>false</skip>
          <enableAssertions>false</enableAssertions>
          <includes>
            <include>**/*IntegrationTest.java</include>
          </includes>
          <systemPropertyVariables>
            <integration>${integration}</integration>
          </systemPropertyVariables>
        </configuration>
      </execution>
    </executions>
</plugin>

Upvotes: -1

user944849
user944849

Reputation: 14951

Since you are running mvn clean install both the integration-test and verify phases should be executing. According to the failsafe plugin docs the failsafe:integration-test and failsafe:verify goals are bound to those phases, so I don't believe the extra call to failsafe:integration-test is required.

That said however, I'm not sure I trust the failsafe plugin documentation. I answered a similar question for someone earlier this year. It turned out he had to explicitly bind each goal to the correct phase and then failsafe worked as expected. Might be worth a shot.

Upvotes: 3

Related Questions