CODEWITHSUNDEEP
androidandroid-testingcucumber-jvm
André Diermann
André Diermann

Reputation: 2783

How to continue an Android instrumentation test run after exception?

Apparently an instrumentation test run is stopped when on exception occurs in the instrumented application:

Test failed to run to completion. Reason: 'Instrumentation run failed due to 'Exception''. Check device logcat for details

Is this the desired behavior or a misconfiguration in a custom instrumentation runner?

I'm using a custom MonitorinInstrumentation [1] in order to automate acceptance tests.

Unfortunately test execution is canceled when on exception occurs in one test. I want the test suite to complete and only mark the failed tests, like in JUnit.

Is there a general approach to execute (connected) tests without quitting the whole instrumentation in case an exception occurs in one test?

  1. https://github.com/cucumber/cucumber-jvm/blob/master/examples/android/android-studio/Cukeulator/app/src/androidTest/java/cucumber/cukeulator/test/Instrumentation.java

Upvotes: 18

Views: 6573

Answers (4)

张小贤
张小贤

Reputation: 41

You can try Android Test Orchestrator (I didn't try it.).

According to the documentation:

Crashes are isolated: Even if one test crashes, it takes down only its own instance of Instrumentation. This means that the other tests in your suite still run, providing complete test results.

See Android Test Orchestrator

Upvotes: 0

whla
whla

Reputation: 811

I was recently looking for a solution to this problem and found some info that other might find useful.

Enter org.junit.rules.ErrorCollector. While this won't work if your instrumentation tests are using deprecated ApplicationTestCase<Application> class, I was able to switch to using androidx.test.platform.app.InstrumentationRegistry to get around this issue since I only needed the application context.

Here is some example code from a gist I found to help get started with:

package com.foo;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ErrorCollector;

import static org.hamcrest.Matchers.equalTo;

/**
 * Created: 12/04/19 17:12
 *
 * @author chris
 */
public class MyTest {
    @Rule
    public ErrorCollector collector = new ErrorCollector();

    @Test
    public void myTest() {
        collector.checkThat("a", equalTo("b"));
        collector.checkThat(1, equalTo(2));
    }
}

Upvotes: 0

newhouse
newhouse

Reputation: 946

You can try to put in your root build.gradle

tasks.withType(Test) {
  ignoreFailures = true
}

According to the documentation:

If one of the test fails, on any device, the build will fail.

Check this android testing

Upvotes: 1

Lorenzo Camaione
Lorenzo Camaione

Reputation: 525

Instrumentation tests raise an Exception when something goes wrong (e.g. some conditions that you want to check). You can usually avoid some test to fail using try catch statement(or changing those checks). In this case there's something that made Dalvik Virtual Machine stop. This is usually caused by a crash in your app. Try to check carefully your tests flow to analyze if there are some crashes. Also, be sure to not use System.exit(0) in onDestroy() in some of your activities because this can cause your problem. I hope to can help you.

Upvotes: 2

Related Questions