Patrick Rode
Patrick Rode

Reputation: 339

How to retry playwrigth tests with Java?

We are using playwrigth for some UI tests which are flaky. So I want to retry each test a couple of times before marking it as failed. So I created my own JUnit 5 extension. But unfortunatelly it doesn't work together with playwrigth, because of the parameter of the test functions.

@Retention(RetentionPolicy.RUNTIME)
public @interface Retry {
    int value() default 5;
}

public class RetryExtension implements TestExecutionExceptionHandler {

    private final AtomicInteger counter = new AtomicInteger(1);

    private void printError(Throwable e) {
        System.err.println(
            "Attempt test execution #" + counter.get() +
                " failed (" + e.getClass().getName() +
                "thrown):  " + e.getMessage());
    }

    @Override
    public void handleTestExecutionException(
        ExtensionContext extensionContext, Throwable throwable)
        throws Throwable {

        printError(throwable);

        extensionContext.getTestMethod().ifPresent(method -> {
            int maxExecutions =
                method.getAnnotation(Retry.class) != null ?
                    method.getAnnotation(Retry.class).value() : 1;

            while (counter.incrementAndGet() <= maxExecutions) {
                try {
                    extensionContext.getExecutableInvoker().invoke(
                        method,
                        extensionContext.getRequiredTestInstance());
                    return;
                } catch (Throwable t) {
                    printError(t);

                    if (counter.get() >= maxExecutions) {
                        throw t;
                    }
                }
            }
        });
    }
}

@UsePlaywright(DefaultOptions.class)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class E2ETest {
    
    @ExtendWith(RetryExtension.class)
    @Retry
    @Test
    void test(Page page) {
        fail();
    }
}

And here is my stacktrace when I execute the test:

Attempt test execution #1 failed (org.opentest4j.AssertionFailedErrorthrown):  
Attempt test execution #2 failed (org.junit.jupiter.api.extension.ParameterResolutionExceptionthrown):  No ParameterResolver registered for parameter [com.microsoft.playwright.Page page] in method [void E2ETest.test(com.microsoft.playwright.Page)].
Attempt test execution #3 failed (org.junit.jupiter.api.extension.ParameterResolutionExceptionthrown):  No ParameterResolver registered for parameter [com.microsoft.playwright.Page page] in method [void E2ETest.test(com.microsoft.playwright.Page)].
Attempt test execution #4 failed (org.junit.jupiter.api.extension.ParameterResolutionExceptionthrown):  No ParameterResolver registered for parameter [com.microsoft.playwright.Page page] in method [void E2ETest.test(com.microsoft.playwright.Page)].
Attempt test execution #5 failed (org.junit.jupiter.api.extension.ParameterResolutionExceptionthrown):  No ParameterResolver registered for parameter [com.microsoft.playwright.Page page] in method [void E2ETest.test(com.microsoft.playwright.Page)].

org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter [com.microsoft.playwright.Page page] in method [void E2ETest.test(com.microsoft.playwright.Page)].

    at RetryExtension.lambda$handleTestExecutionException$0(RetryExtension.java:33)
    at java.base/java.util.Optional.ifPresent(Optional.java:178)
    at RetryExtension.handleTestExecutionException(RetryExtension.java:26)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Disconnected from the target VM, address: '127.0.0.1:55390', transport: 'socket'

Process finished with exit code -1

So only in the first attempt I really got into the test method with a valid Page object as parameter. If I use the retry extension for tests without parameters, it works fine. But I need the Page given by playwrigth and don't know how I can combine those two worlds.

Upvotes: 1

Views: 30

Answers (0)

Related Questions