CODEWITHSUNDEEP
testng
Siva
Siva

Reputation: 13

Testng Test Execution not happening in alphabetical order

I have test class with the below @Test methods

@Test
public void Submission01(){
}

@Test (dependsOnMethods = "Submission01")
public void Submission02(){

}

@Test
public void Onboarding1(){

}

@Test (dependsOnMethods="Onboarding1")
public void Onboarding2(){
}

With these methods when i execute the class, the testng executes in the below order 

Onboarding1
Submission01
Onboarding2
Submission02

I'm expecting to execute in 

Onboarding1
Onboarding2
Submission01
Submission02

Please let me know what is the issue and how to execute the above in my expected order

Upvotes: 0

Views: 1673

Answers (5)

Krishnan Mahadevan
Krishnan Mahadevan

Reputation: 14746

I am just going to be adding a bit more information to what @Mitul Lakhani has explained as one of the answers for this question (https://stackoverflow.com/a/53060920/679824)

TestNG relies on reflection. The Java Reflection APIs does not guarantee the method order when we use it to introspect a class to find out what are the test methods that are available in it. So the order of independent methods (Methods that dont have either soft or hard dependency) execution is never guaranteed.

  • Soft dependency - This is usually achieved in TestNG by using the priority attribute for the @Test annotation. Its called a soft dependency because TestNG will continue to execute all the methods even though a previous method with a higher priority failed.
  • Hard dependency - This is usually achieved in TestNG by using either dependsOnMethods (or) dependsOnGroups attribute for the @Test annotation. It's called a hard dependency because TestNG will continue to execute a downstream method if and only if an upstream method ran successfully.

Now assuming that your class has only independent methods (that is you have not used any of the ordering attributes viz., priority/dependsOnMethods/dependsOnGroups, then you can basically leverage a TestNG listener for ordering the methods.

You do this by implementing the interface org.testng.IMethodInterceptor and then wiring in the implementation via one of the following means :

  1. @Listeners annotation
  2. <listeners> tag
  3. Service loading capabilities of TestNG

For more information on how to work with listeners in general please refer to my blogpost here

Here's a full fledged example that shows this in action

import com.rationaleemotions.stackoverflow.qn53059530.ExampleTest.MethodOrder;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import org.testng.IMethodInstance;
import org.testng.IMethodInterceptor;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(MethodOrder.class)
public class ExampleTest {
  @Test
  public void Submission01() {
    print();
  }

  @Test
  public void Submission02() {
    print();
  }

  @Test
  public void Onboarding1() {
    print();
  }

  @Test
  public void Onboarding2() {
    print();
  }

  private void print() {
    ITestResult current = Reporter.getCurrentTestResult();
    System.err.println("Executing :" + current.getMethod().getMethodName() + "()");
  }

  public static class MethodOrder implements IMethodInterceptor {

    @Override
    public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
      List<IMethodInstance> ordered = new ArrayList<>(methods);
      ordered.sort(Comparator.comparing(o -> o.getMethod().getMethodName()));
      return ordered;
    }
  }
}

Output is as below

Executing :Onboarding1()
Executing :Onboarding2()
Executing :Submission01()
Executing :Submission02()

===============================================
Default Suite
Total tests run: 4, Passes: 4, Failures: 0, Skips: 0
===============================================

The example as you would have noticed neither uses priority nor dependsOnMethods/dependsOnGroups.

You can tweak the listener implementation to decide any way in which you want the ordering to be (chronological order (or) reverse chronological order for e.g.,)

Upvotes: 4

BhavinD.
BhavinD.

Reputation: 481

If I understand, your code having some syntax error. You miss to put curly brackets in TestNG parameter (dependsOnMethods = "Submission01") It should be like (dependsOnMethods = {"Submission01"}).

Hope, below sample code works for you.

@Test
public void Submission01(){
}

@Test (dependsOnMethods = {"Submission01"})
public void Submission02(){

}

@Test
public void Onboarding1(){

}

@Test (dependsOnMethods = {"Onboarding1"})
public void Onboarding2(){
}

Upvotes: 0

Waqar Nadir
Waqar Nadir

Reputation: 388

if you are using testNG then you can use priority parameter for this.

@Test(priority=1)    
public void Onboarding1(){    
}

@Test (dependsOnMethods="Onboarding1",priority=2)    
public void Onboarding2(){
}

@Test(priority=3)    
public void Submission01(){
}

@Test (dependsOnMethods = "Submission01",priority=4)    
public void Submission02(){
}

Upvotes: 0

Java_Help_Line
Java_Help_Line

Reputation: 34

Try this...

@Test(dependsOnMethods = "Onboarding2")
public void Submission01{
}

@Test (dependsOnMethods = "Submission01")
public void Submission02(){
}

@Test
public void Onboarding1(){
}

@Test (dependsOnMethods="Onboarding1")
public void Onboarding2(){
}

Upvotes: 0

Mitul Lakhani
Mitul Lakhani

Reputation: 74

Here's how TestNG works with respect to @Test methods.

First it executes all "independent" methods.(If there is ambiguity in this ordering uses priority to resolve ambiguity for independent methods)

Then it executes all methods that have a "dependsOnGroups/dependsOnMethods" attribute.(If there is ambiguity in this ordering uses priority to resolve ambiguity for independent methods)

If there is still ambiguity because of no priority or same priority, it executes in alphabetical order

Upvotes: 2

Related Questions