How to run Cucumber test on Android

Question

Similar questions have been asked but the problem was always either an old version of the tools or a mix of android.support and androidx libraries.

I have the latest version of AS and I'm using the version 6.1.1 of the gradle-wrapper and the version 4.0.1 of gradle. I have installed the plugins from JetBrains for Gherkin (version 193.6911.14) and Cucumber (version 193.6494.1)

I created a sample app using the AS wizard to build an app with an empty activity and then I have modified the build.gradle file:

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'

android {
    compileSdkVersion 30
    buildToolsVersion "29.0.3"

    defaultConfig {
        applicationId "com.sampleapp.cucumber"
        minSdkVersion 21
        targetSdkVersion 30
        versionCode 1
        versionName "1.0"
        multiDexEnabled true
        testApplicationId "com.sampleapp.cucumber.test"
        testInstrumentationRunner "io.cucumber.android.runner.CucumberAndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }

    sourceSets {
        androidTest {
            assets.srcDirs = ['src/androidTest/assets']
        }
    }

    testOptions {
        execution 'ANDROIDX_TEST_ORCHESTRATOR'
    }
}


dependencies {
    implementation Kotlin.stdlib
    implementation Kotlin.ktxCore
    implementation AndroidSupport.appCompat
    implementation AndroidSupport.constraintLayout
    androidTestImplementation AndroidTest.rules
    androidTestImplementation AndroidTest.ext
    androidTestImplementation AndroidTest.runner
    androidTestImplementation AndroidTest.cucumber
    androidTestImplementation AndroidTest.espressoCore
    androidTestImplementation AndroidTest.orchestrator

}

class AndroidTest {
    public static rules = "androidx.test:rules:${Version.testCore}"
    public static runner = "androidx.test:runner:${Version.testCore}"
    public static ext = "androidx.test.ext:junit:${Version.testExt}"
    public static espressoCore = "androidx.test.espresso:espresso-core:${Version.espresso}"
    public static orchestrator = "androidx.test:orchestrator:${Version.orchestrator}"
    public static cucumber = "io.cucumber:cucumber-android:${Version.cucumber}"
}

class Kotlin {
    public static stdlib = "org.jetbrains.kotlin:kotlin-stdlib:${Version.kotlin}"
    public static ktxCore = "androidx.core:core-ktx:${Version.ktx}"
}

class AndroidSupport {
    public static appCompat = "androidx.appcompat:appcompat:${Version.appCompat}"
    public static constraintLayout = "androidx.constraintlayout:constraintlayout:${Version.constraintLayout}"
}


class Version {
    public static testCore = "1.3.0"
    public static testExt= "1.1.2"
    public static espresso = "3.3.0"
    public static cucumber = "4.3.0"
    public static orchestrator = "1.1.0"
    public static appCompat = "1.1.0"
    public static constraintLayout = "1.1.3"
    public static kotlin = "1.4.0"
    public static ktx = "1.3.0"

}

I have added:

@Suppress("unused")
@CucumberOptions(
        features = [
            "features"
        ],
        glue = ["com.sampleapp.cucumber.test"],
        tags = ["@android", "~@manual"],
)
class CucumberOptionClass

and the feature file:

Feature: Feature description

  @ios @android
  Scenario: Scenario description
    Given Something is true
    When Something happens
    Then Something is true and something happens

the steps' definition:

class SomeTestImplementation {

    private lateinit var context: Context

    @Before
    fun setUp() {
        context = InstrumentationRegistry.getInstrumentation().targetContext
    }

    @Given("Something is true")
    fun somethingIsTrue() {
    }

    @When("Something happens")
    fun somethingHappens() {
    }

    @Then("Something is true and something happens")
    fun somethingIsTrueAndSomethingHappens() {
        assertTrue(false)
    }
}

This is the structure of the folder

--main
    --java
        --app.package
            MainActivity
--androidTest
    --java
        --app.package.test 
            --steps
                --example
                    SomeTestImplementation
            CucumberOptionClass
    --assets
        --features
            Sample.features

When I run ./gradlew clean connectedCheck I get the error:

Starting 0 tests on [ANA-NX9 - 10] Tests on [ANA-NX9 - 10] failed: No test results

com.android.build.gradle.internal.testing.ConnectedDevice > No tests found.[ANA-NX9 - 10] FAILED No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).

If I run the scenario in Android Studio (right-clicking on the Scenario and choosing Run) I get a different error:

java.lang.IllegalStateException: No instrumentation registered! Must run under a registering instrumentation. at androidx.test.platform.app.InstrumentationRegistry.getInstrumentation(InstrumentationRegistry.java:45) at com.sampleapp.cucumber.test.steps.example.SomeTestImplementation.setUp(SomeTestImplementation.kt:16) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at cucumber.runtime.Utils$1.call(Utils.java:26) at cucumber.runtime.Timeout.timeout(Timeout.java:16) at cucumber.runtime.Utils.invoke(Utils.java:20) at cucumber.runtime.java.JavaHookDefinition.execute(JavaHookDefinition.java:60) at cucumber.runner.HookDefinitionMatch.runStep(HookDefinitionMatch.java:16) at cucumber.runner.TestStep.executeStep(TestStep.java:65) at cucumber.runner.TestStep.run(TestStep.java:50) at cucumber.runner.TestCase.run(TestCase.java:42) at cucumber.runner.Runner.runPickle(Runner.java:49) at cucumber.runtime.Runtime$1.run(Runtime.java:82) at cucumber.runtime.Runtime$SameThreadExecutorService.execute(Runtime.java:217) at cucumber.runtime.Runtime.run(Runtime.java:79) at cucumber.api.cli.Main.run(Main.java:26) at cucumber.api.cli.Main.main(Main.java:8)

If I remove the line

context = InstrumentationRegistry.getInstrumentation().targetContext

In the steps' definition then I still get the same error from the command line but the test runs successfully when I run it in AS (it shows the failure for the last step but that is expected).

If I replace the Runner with the AndroidJunitRunner and I add a non Cucumber test, then everything works: I can fetch the context as well.

If from the command line I follow the steps:

./gradlew assemble
./gradlew assembleAndroidTest
adb install ./app/build/outputs/apk/debug/app-debug.apk  
adb install ./app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
adb shell pm list instrumentation | grep sampleapp

I can see

instrumentation:com.sampleapp.cucumber.test/io.cucumber.android.runner.CucumberAndroidJUnitRunner (target=com.sampleapp.cucumber)

but when I run

adb shell am instrument -w com.sampleapp.cucumber.test/io.cucumber.android.runner.CucumberAndroidJUnitRunner

I see the error:

INSTRUMENTATION_RESULT: shortMsg=Process crashed. INSTRUMENTATION_CODE: 0

I have also tried to remove the tags to reduce the issues with finding the test when running on the CLI

How to run Cucumber test on Android

Answers (1)

Related Questions