CODEWITHSUNDEEP
junit
mrQWERTY
mrQWERTY

Reputation: 4149

JUnit - How to make test case pass if it times out?

I ran into a dilemma with making a test pass if it times out.

@Test(timeout=1, expected=Exception.class)
public void testMovesToSolveMaximum() {
    PuzzleSolver pS = createSimplePuzzleSolver(maximumPuzzleStateA, maximumPuzzleStateB);
    PuzzleState goal = new SimplePuzzleState();
    goal.configureState(maximumPuzzleStateB);
    checkThatComputedSolutionIsCorrect(pS, goal);
}

However, the test case fails due to timeout even though I specified that is the expected result.

Upvotes: 2

Views: 872

Answers (1)

shlomi33
shlomi33

Reputation: 1568

If I understand the question correctly then you are observing the specific behavior due to the way that the default JUnit runner is evaluating the whole test:

After realizing that there is a timeout set on your test method it runs it in a different thread and is waiting for the result. As the timeout in your example is set to 1[ms] I believe that it reaches the timeout before the test actually finishes which makes the runner throw the timeout exception (that is indeed a java.lang.Exception) which you thought needed to be caught by the expected attribute in the Test annotation. But the attribute expected on the Test annotation is evaluating only the exceptions thrown from the test method and not from the timeout checking mechanism. In other words the expected exception mechanism is not working for the timeout exception throw by the f/w and not by a test.

You can explore this yourself starting in BlockJUnit4ClassRunner class in JUnit (relevant part to start from. NOTE: it is not so easy to go over the code and understand the flow...):

protected Statement methodBlock(FrameworkMethod method) {
    Object test;
    try {
        test = new ReflectiveCallable() {
            @Override
            protected Object runReflectiveCall() throws Throwable {
                return createTest();
            }
        }.run();
    } catch (Throwable e) {
        return new Fail(e);
    }

    Statement statement = methodInvoker(method, test);
    statement = possiblyExpectingExceptions(method, test, statement);
    statement = withPotentialTimeout(method, test, statement);
    statement = withBefores(method, test, statement);
    statement = withAfters(method, test, statement);
    statement = withRules(method, test, statement);
    return statement;
}

Upvotes: 4

Related Questions