CODEWITHSUNDEEP
javatestingtestngintegration-testing
user1810502
user1810502

Reputation: 529

TestNG: RetryAnalyzer, dependent groups are skipped if a test succeeds upon retry

I have a RetryAnalyzer and RetryListener. In RetryListener onTestFailure, I check if the test is retryable, if yes I set the result to SUCCESS. I also do, testResult.getTestContext().getFailedMethods.removeResult(testResult) in this method.

I again remove failed results (with valid if conditions) in onFinish method in the listener.

Now the problem I am running into is, I made each test class into groups. One test class does the WRITES and one test class does the READS. So READs group depends on WRITES.

If a test case fails on 1st attempts and succeeds on retrying, then all the test cases in the dependent group are SKIPPED, despite removing failed result in onTestFailure method.

Is there a way to run dependent method if a test case succeeds on retrying? I am fine with the behavior if the test case fails in all attempts, so I am not looking to add "alwaysRun=true" on each dependent method.

Upvotes: 2

Views: 2129

Answers (3)

Umang Gupta
Umang Gupta

Reputation: 1

In retry file, add a mechanism to see if retry is left of the case.

In Custom Listener, override onTestSkipped() and check if RetryLeft, remove it from skippedResult and return

public class Retry implements IRetryAnalyzer {

   private int count = 0;
   private static final List retriedTests = new CopyOnWriteArrayList();
   private static final ConcurrentHashMap<String, Boolean> retriedTestsMap = new ConcurrentHashMap();

   @Override
   public boolean retry(ITestResult iTestResult) {
      int maxTry = 3;
      if (!iTestResult.isSuccess()) { // Check if test not succeed
         String name = getNameForTestResult(iTestResult);
         if (count < maxTry) { // Check if maxTry count is reached
            count++; // Increase the count count by 1
            retriedTests.add(iTestResult);
            retriedTestsMap.put(name, true);
            RestApiUtil.println("**" + name + " retry count " + count + " **");
            iTestResult.setStatus(ITestResult.FAILURE); // Mark test as failed
            return true; // Tells TestNG to re-run the test
         } else {
            iTestResult.setStatus(ITestResult.FAILURE); // If maxCount reached,test marked as failed
            retriedTestsMap.put(name, true);
         }
      } else {
         iTestResult.setStatus(ITestResult.SUCCESS); // If test passes, TestNG marks it as passed
      }
      return false;
   }

   public static List getRetriedTests() {
      return retriedTests;
   }

   public static boolean isRetryLeft(ITestResult tr) {
      return retriedTestsMap.get(getNameForTestResult(tr));
   }

   private static String getNameForTestResult(ITestResult tr) {
      return tr.getTestClass().getRealClass().getSimpleName() + "::" + tr.getName();
   }

}





public class CustomTestNGListener extends TestListenerAdapter {
   @Override
   public void onTestSkipped(ITestResult tr) {
      if (Retry.isRetryLeft(tr)) {
         tr.getTestContext().getSkippedTests().removeResult(tr);
         return;
      }
      super.onTestSkipped(tr);
   }

}

Upvotes: 0

Gowtam Dora Sunkavalli
Gowtam Dora Sunkavalli

Reputation: 31

On retry you should be removing the test from the Failed tests. And plz be sure to remove ITestResult object. (i.e, result but not result.getMethod())

@Override
public boolean retry(ITestResult result) {
    if (currentCount < maxRetryCount) {
        result.getTestContext().getFailedTests().removeResult(result);
        currentCount++;
        return true;
    }
    return false;
}

Upvotes: 3

user1810502
user1810502

Reputation: 529

I was using TestNG 6.8.7, upgraded it to 6.9.5.

After that, upon retry, TestNG was marking test case as SKIPPED. I just had to create a Listener, which implemented TestListenerAdapter and override onTestSkipped, if there are retries available then remove the method from skippedTests.

result.getTestContext().getSkippedTests().removeResult(result.getMethod());

If not set test to FAILURE. Now it works as expected.

Upvotes: 2

Related Questions