CODEWITHSUNDEEP
androidamazon-web-servicesandroid-espresso
Lorenzo Camaione
Lorenzo Camaione

Reputation: 525

AWS Device Farm Espresso tests order

I'm starting using Device Farm. My problem is that it is ignoring the order that I used to write tests. In local all works fine because Espresso executes tests in a certain order. To declare that order I used alphabetical order. All my classes starts with a letter (E.g. "A_EspressoTest") so I can choose which class has to be ran first.

Into my classes I use 

@FixMethodOrder(MethodSorters.NAME_ASCENDING)

to declare in which order my tests have to be ran.

It seems also like Device Farm ignores all my annotations (E.g. "@Test" ) because it is running also methods that doesn't have that annotation.

Upvotes: 0

Views: 239

Answers (1)

ahawker
ahawker

Reputation: 3374

Lorenzo,

As of today, there is no way to specify the order of test execution with Espresso. Additionally, your observation about the @Test annotation is correct, we don't currently use that when discovering what test classes/test methods are selected.

AWS Device Farm currently discovers tests based on JUnit 3 style naming conventions (classes starting/ending with the word Test and methods within those classes starting with the word test.

For example:

// This class would be automatically discovered (by name).
public class LoginTests extends InstrumentationTestCase {
    // This method would be automatically discovered (by name).
    public void testLoginFormUsingInvalidCredentials() {
        // ...
    }

    // This method would **not** be automatically discovered.
    @Test
    public void loginWithValidCredentials() {
        // ...
    }
}

// This class would **not** be automatically discovered.
public class Login extends InstrumentationTestCase {
    // This method would **not** be automatically discovered since the class was not discovered.
    public void testLoginFormWithValidCredentials() {
        // ...
    }
}

With all that said, we've heard plenty of feedback and requests for supporting test discovery using all JUnit 4 annotations and it's an area of improvement we're definitely taking a look at.

Hope that helps!

Best,

Andrew @ AWS Device Farm

Upvotes: 1

Related Questions