CODEWITHSUNDEEP
javaspring-bootselenium-webdrivercucumber
kakemonsteret
kakemonsteret

Reputation: 167

Selenium and Cucumber in a Java/SpringBoot enironment - automatic retry of failed tests

I've got a rather large test suite with Selenium/Cucumber, which runs in a Java/Spring-Boot/Angular environment.

The problem is the classical one: Flaky tests, due to page hickups and/or server/environment issues.

I know there is a way to set cucumber to retry failed tests a number of times. I've seen some rather advanced solutions, with the creatin of a failed tests list, then rerunning these afterwards, etc.

But the I came across this, which is said to be available in newer Cucumber versions:

cucumber --retry 2

The questions is: How to implement this in my setup?

Here's my CucumberTest.java file, where plugins, location of features etc. are specified:

Slf4j
@RunWith(Cucumber.class)
@CucumberOptions(
    plugin   = { "pretty", "json:target/cucumber.json", "html:target/cucumber-html-report",  },
    tags = { "~@integration", "~@Ignore", "~@ToggledOff", "~@Known_error" },
    features = { "src/test/resources/features" })

public class CucumberTest {

@BeforeClass
public static void setup() {
    log.info("\n\n-------------------------------------------------------\n" +
        " RUN ALL SCENARIOS FOR System\n" +
        "\n-------------------------------------------------------\n"
    );

}

@AfterClass
public static void quitTestSuite() {
    Hooks.closeConnections();
    Hooks.stopWatchTotalTime.stop();
    StringBuilder slowTests = findPotensialSlowTests();
    log.info("\n\n-------------------------------------------------------" +
            "\nALL SCENARIOS FINISHED FOR SYSTEM" +
            "\nTotal time         : {}" +
            "\nNumber of scenarios: {}" +
            slowTests.toString() +
            "\n-------------------------------------------------------\n"
        , Hooks.stopWatchTotalTime, Hooks.scenarioCounter.get());

}

Any ideas? Is there a place for that (and potentially other) cucumber option in this file? Or somewhere else? All help/hints are appreciated.

UPDATE: Silly question, but is "cucumber-jvm" named "cucumber-java" in the config files and other places? I cannot find the line cucumber-jvm anywhere, but I do find cucumber-java.

Upvotes: 0

Views: 687

Answers (2)

M.P. Korstanje
M.P. Korstanje

Reputation: 12039

Cucumber-JVM doesn't support retry.

However if you use Cucumber in combination with Maven you can use:

mvn -Dsurefire.rerunFailingTestsCount=2 test

Do note that this will effectively run Cucumber twice. So any reports generated by Cucumber will be overwritten by the rerun.

Also note that the reports generated by Maven aren't great. They get better if you use Cucumber in combination with the Cucumber JUnit Platform Engine (JUnit 5).

Upvotes: 1

Akzy
Akzy

Reputation: 1928

You can add retry = 2 line after the below line in your CucumberTest.java file inside the @CucumberOptions annotation

features = { "src/test/resources/features" },

Upvotes: 1

Related Questions