CODEWITHSUNDEEP
mavenjunitintegration-testingmaven-failsafe-plugin
Dave
Dave

Reputation: 19090

How do I control order of test execution within a single integration test file?

I’m using Maven 3.0.3, Failsafe plugin v2.17 and JUnit 4.11. Currently I have an integration test with tests in the following order

@RunWith(SpringJUnit4ClassRunner.class)
public class MyTests {

    @Test
    public final void testAdd() {
        …
    }

    @Test
    public final void testUpdate() {
        …
    }

    @Test
    public final void testDelete() {
        …
    }

Currently when I run the tests through Maven as part of a “mvn clean install” run, the “testDelete” is getting run before the “testAdd” or “testUpdate”. If I change the name to “testZZZDelete”, then it gets run last but I don’t like that.

How do I get the tests to run in the order that I specify them in the file? My failsafe configuration is like so:

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.17</version>
        <configuration>
                <reuseForks>true</reuseForks>
                <argLine>-Xmx4096m -XX:MaxPermSize=512M ${itCoverageAgent}</argLine>
        </configuration>
        <executions>
                <execution>
                        <goals>
                                <goal>integration-test</goal>
                                <goal>verify</goal>
                        </goals>
                </execution>
        </executions>
</plugin>

Upvotes: 1

Views: 8923

Answers (4)

isalgueiro
isalgueiro

Reputation: 2033

This JUnit patch implements this feature https://github.com/adko-pl/junit/commit/4421261dbdcaed8ff0a82f4d5229ac8ad6c97543, thought it seems that the author didn't add the order param to the org.junit.Test annotation, using a patched JUnit will be way more easy than adapting all your current tests in a different framework IMHO.

See How to run test methods in specific order in JUnit4?

Upvotes: 0

Matthew Farwell
Matthew Farwell

Reputation: 61695

Short answer: no, there isn't anyway apart from renaming the tests so that they execute in order. You can use the @FixMethodOrder(NAME_ASCENDING) annotation to ensure that the tests are executed in alphabetical order.

Long answer: As I'm sure you know, surefire/failsafe gives you the option of ordering test classes through the runOrder configuration parameter. This controls the order in which each test class is executed. So you can run class foobar.Test1 before foobar.Test2 or the other way around.

For the order of execution of methods within a class, the problem that you are facing is that the JVM doesn't return the list of methods in the same order that they are declared in the file. With Java 6, the order in which they were returned was usually the order of declaration, but this changed with java 7. So, with the release of JUnit 4.11, the default ordering was changed to be based on the hash of the method name, to give a deterministic, but not predictable ordering. This is why you are getting testDelete run before anything else.

After a long discussion, we added the FixMethodOrder annotation to JUnit 4.11 to allow someone to at least be able to rename their methods. This seems to work with the SpringJUnit4ClassRunner - at least with the latest version 4.1.0.RELEASE. I haven't tried other releases.

So, to have a predictable ordering, you could, where necessary, rename your methods to be executed in the order you want and add the @FixMethodOrder annotation to the class.

@RunWith(SpringJUnit4ClassRunner.class)
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class MyTests {

@Test
public final void step1Add() {
    …
}

@Test
public final void step2Update() {
    …
}

@Test
public final void step3Delete() {
    …
}

For more information, please see Has JUnit4 begun supporting ordering of test? Is it intentional?.

Upvotes: 6

Praveen
Praveen

Reputation: 1485

I think TestNg gives an option to execute the tests in user-defined sequence. 

@Test(dependsOnMethods='testAdd')
public final void testAdd() {
    …
}

@Test(dependsOnMethods='testAdd')
public final void testUpdate() {
    …
}

@Test(dependsOnMethods='testUpdate')
public final void testDelete() {
    …
}

So, if the compiler tries to execute the "testUpdate" test, there is a restriction added to the execution. The testUpdate function is depending on the "testAdd" function. So Compiler will first perform the "testAdd", then proceed with the "testUpdate" function.

Hope the tests should be migrated from JUNIT to TestNg.

Upvotes: 1

Paul Harris
Paul Harris

Reputation: 5809

First off, having tests depend on each other is typically bad practice, but i realise this is a purest view and that if it is really so slow to get to that state that you have to continue on then I would say you should have a look at the answers provided over on this post personally I would pick the second option as annotations make it nice and explicit.

Upvotes: -1

Related Questions