Rerun whole class in case of failed test case using TestNG

Question

I have a script setup developed using Appium and TestNG. The TestNG xml contains execution of multiple classes and each class has multiple test cases.

Example:

Class1:
-Class1_Test1
-Class1_Test2
-Class1_Test3
Class2:
-Class2_Test1
-Class2_Test2
-Class2_Test3

I tried integrating IRetryAnalyzer but that just calls the failed test case. The requirement is to execute the complete Class1 in case Class1_Test2 fails as soon as Class1 fails before it proceeds to Class2?

The reason for the ask is the app is a media player and if in case media playback fails due to network/server issues, next test cases of forward and rewind will not be required to be executed and it will need to relaunch the app and retry all steps before performing further tests.

Answer 1

Finally found a workaround to rerun the whole class. I would call it a workaround since technically TestNG does not provide a way to re-execute @BeforeTest in case a failure occurs at any point of time.

The best possible method I found was to have no @BeforeTest section and have just one @Test section and have all Test cases as functions which would be called within the single @Test defined. So in case a failure occured at any point of time, the @Test would be recalled which contains all the functions in the order required including setting up the capabilities. The retry logic reruns the entire @Test section as soon as a failure is observed.

Example:

Before the changes:

package <yourpackagename>;

<import required packages>

public class Home {
    private AppiumDriver<?> driver;
    private static final String url = "http://0.0.0.0:4723/wd/hub";

    <define your variables>

    @Parameters({"deviceOS", "DSN"})
    @BeforeTest
    public void setUp(String deviceOS, String DSN) throws InterruptedException, MalformedURLException, ParseException {
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.BROWSER_NAME, "");
        capabilities.setCapability("deviceName", "FireTVStick");
        capabilities.setCapability("platformVersion", deviceOS);
        capabilities.setCapability("udid", DSN);
        capabilities.setCapability("platformName", "Android");
        capabilities.setCapability("noReset", true);
        capabilities.setCapability("fullReset", false);
        capabilities.setCapability(MobileCapabilityType.NEW_COMMAND_TIMEOUT, 1200);
        capabilities.setCapability("appPackage", "<your app package>");
        capabilities.setCapability("appActivity", "<your launcher activity>");
        driver = new AndroidDriver<>(new URL(url), capabilities);
        driver.manage().timeouts().implicitlyWait(3, TimeUnit.SECONDS);
        //End of Launch Code

        System.out.println("-Testing Home Section-");
    }   


    @Parameters({"DSN"})
    @Test
    public void Test1_VideoPlaybackVerification(String DSN) throws InterruptedException, ParseException{

        //Video playback verification code starts
        .
        .
        //End of code for Video playback verification
    }


    @Test //Test Case for Pause verification
    public void Test2_PauseVerification() throws InterruptedException, ParseException{
        //Video pause verification code starts
        .
        .
        //End of code for Video pause verification
    }

    @AfterTest
    public void End() {
        driver.quit();
    }
}

After the changes:

package <yourpackagename>;

<import required packages>

@Listeners(MyTestListenerAdapter.class)
public class Home {
    private AppiumDriver<?> driver;

    <define your variables>

    public void setUp(String port, String deviceOS, String DSN, String deviceName) throws InterruptedException, MalformedURLException {
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.BROWSER_NAME, "");
        capabilities.setCapability("platformVersion", deviceOS);
        capabilities.setCapability("deviceName", deviceName);
        capabilities.setCapability("udid", DSN);
        capabilities.setCapability("platformName", "Android");
        capabilities.setCapability("noReset", true);
        capabilities.setCapability("fullReset", false);
        capabilities.setCapability(MobileCapabilityType.NEW_COMMAND_TIMEOUT, 1200);
        capabilities.setCapability("appPackage", "<your app package>");
        capabilities.setCapability("appActivity", "<your launcher activity>");
        final String url = "http://127.0.0.1:"+port+"/wd/hub";
        driver = new AndroidDriver<>(new URL(url), capabilities);
        driver.manage().timeouts().implicitlyWait(3, TimeUnit.SECONDS);
    }

    public void HomeVerification(String DSN, String deviceName) throws InterruptedException, ParseException
    {
        System.out.println(deviceName+": Testing Home Section-");

        <--Your code to perform any additional task before execution-->
    }

    public void Test1_VideoPlaybackVerification(String DSN, String deviceName) throws InterruptedException, ParseException
    {       
        //Video playback verification code starts
        .
        .
        //End of code for Video playback verification
    }

    public void Test2_PauseVerification(String deviceName) throws InterruptedException, ParseException
    {

        //Video pause verification code starts
        .
        .
        //End of code for Video pause verification
    }

    @Parameters({"port", "deviceOS", "DSN", "deviceName"})
    @Test (retryAnalyzer = Retry.class)
    public void TestRun(String port, String deviceOS, String DSN, String deviceName) throws InterruptedException, ParseException, MalformedURLException {

        try {
            setUp(port, deviceOS, DSN, deviceName);
            HomeVerification(DSN, deviceName);
            Test1_VideoPlaybackVerification(DSN, deviceName);
            Test2_PauseVerification(deviceName);
        } catch (WebDriverException e) {
            // TODO Auto-generated catch block
            Reporter.log(deviceName+": Error observed while executing script!", true);
            Assert.assertTrue(false); //Fails the test case
        }

    }

    @AfterTest
    public void End() {
        driver.quit();
    }
}

Answer 2

I am using the group test. It will continue the test even if some test fail in the class.

In your class file you can define group like following.

public class myClass(){
    @Test(groups = {"abc"}, priority = 1)
    public void test1(){
    }
    @Test(groups = {"abc"}, priority = 2)
    public void test2(){
    }
    @Test(groups = {"abc"}, priority = 3)
    public void test3(){
    }
}

similarly you can define second class with same group name or different group name. Priority determines the order in which test case runs.

Your Testng.xml file will be look like following:

<test name="any name">
  <groups>
    <run>

      <include name="abc"  />
      <include name="any other group name"  />
    </run>
  </groups>

  <classes>
    <class name="packageName.myClass"/>
    <class name="your_packageName.class2"/>
  </classes>
</test>

packageName is the path where your Test class is located. If your test class and testng.xml files are in same package, packageName is not required.

For more information about test method in Testng refer this link.

Answer 3

There is no way to achieve this as per TestNg documentation, might be below answer can help you

Retry Logic - retry whole class if one tests fails - selenium