Pino
Pino

Reputation: 9333

Tests execution order: JUnit 5 vs Maven Surefire Plugin documentation

I know that tests should not rely on the order in which they are executed but sometimes bad things happen. I have just fixed a Maven project whose tests were passing when executed by Jenkins but not when executed by hand on the command line and now I am examining the documentation to better understand how things work.

JUnit 5 says:

By default, test classes and methods will be ordered using an algorithm that is deterministic but intentionally nonobvious.

and

To control the order in which test methods are executed, annotate your test class or test interface with @TestMethodOrder

On the contrary, Maven says:

<runOrder> Defines the order the tests will be run in. Supported values are alphabetical, reversealphabetical, random, hourly (alphabetical on even hours, reverse alphabetical on odd hours), failedfirst, balanced and filesystem. Default: filesystem.

How can the two things coexist? In my case it is evident that the execution order was not deterministic but it was not filesystem based either (checked with ls -lU). According to Maven log the execution order on the two servers was simply reversed and in both cases it was different from the filesystem order that was alphabetic. Is there any other setting that takes precedence?

Upvotes: 1

Views: 148

Answers (0)

Related Questions