CODEWITHSUNDEEP
javaunit-testingtestng
Classified
Classified

Reputation: 6050

What is controlling the order of group tests in TestNG?

How does TestNG determine the order of running tests in groups when I don't use features like dependsonGroup=X or priority=X?

Long story short, I was looking at this test site:

http://www.tutorialspoint.com/testng/testng_group_test.htm

I cut and pasted the lines for the 3 example files to play around with and try to understand TestNG and group tests. When I ran it, the output matched what the site said.

Good Results:

Inside testPrintMessage()
.com
Inside testExitMessage()
www..com

===============================================
Suite1
Total tests run: 2, Failures: 1, Skips: 0
===============================================

Next thing I did was I thought I was tidying the code and fixing a typo, which gave me a weird result. From GroupTestExample.java, I changed the method from testingExitMessage to testExitMessage. I don't know how 3 letters can change the order of tests being run.

Bad Results - See how the order of the test functions swapped, which affected the number of failures

Inside testExitMessage()
www..com
Inside testPrintMessage()
www..com

===============================================
Suite1
Total tests run: 2, Failures: 2, Skips: 0
===============================================

Can someone explain to me what may have happened, why changing the method name made the order of the test change?

MessageUtil.java

/*
* This class prints the given message on console.
*/
public class MessageUtil {
    private String message;

    // Constructor
    // @param message to be printed
    public MessageUtil(String message) {
        this.message = message;
    }

    // prints the message
    public String printMessage() {
        System.out.println(message);
    return message;
    }

    // add "tutorialspoint" to the message
    public String salutationMessage() {
        message = "tutorialspoint" + message;
    System.out.println(message);
    return message;
    }

    // add "www." to the message
    public String exitMessage() {
    message = "www." + message;
    System.out.println(message);
    return message;
    }
}

Good GroupTestExample.java 

public class GroupTestExample {
    String message = ".com";
    MessageUtil messageUtil = new MessageUtil(message);

    @Test(groups = { "functest", "checkintest" })
    public void testPrintMessage() {
        System.out.println("Inside testPrintMessage()");
    message = ".com";
    Assert.assertEquals(message, messageUtil.printMessage());
    }

    @Test(groups = { "checkintest" })
    public void testSalutationMessage() {
    // Removed code since this method wasn't the problem method
    }

    @Test(groups = { "functest" })
    public void testingExitMessage() {
        System.out.println("Inside testExitMessage()");
        message = "www." + "tutorialspoint"+".com";
    Assert.assertEquals(message, messageUtil.exitMessage());
    }
}

testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Suite1">
    <test name="test1">
        <groups>
        <run>
        <include name="functest" />
        </run>
    </groups>
    <classes>
        <class name="GroupTestExample" />
    </classes>
    </test>
</suite>

"Bad" GroupTestExample.java

    // same code as above except I changed this method from testingExitMessage to
    // testExitMessage b/c that's what the println msg shows and I thought I was 
    // fixing a typo.

    public void testExitMessage() {
        System.out.println("Inside testExitMessage()");
        message = "www." + "tutorialspoint"+".com";
    Assert.assertEquals(message, messageUtil.exitMessage());
    }

Upvotes: 1

Views: 1500

Answers (2)

Marina Touceda
Marina Touceda

Reputation: 21

You can set up a priority in your test.

ex. @Test (priority=1)

Then, in your testNG.xml file you can set preserve-order="true"

Upvotes: 1

YoK
YoK

Reputation: 1636

TestNG run tests in order they found in XML file, in your case you do not not specify methods in XML file and test run in unpredicted order (i believe it depends on compiler).

Upvotes: 2

Related Questions