Android instrumentation run failed - assertion error

Question

I am struggling to figure out why my test cases fail to run to completion. I'm confused because the error appears to show simply that an assertion failed. That means the test is failing, but why should that crash the entire test runner?

I am seeing this error in the console.

Test failed to run to completion. Reason: 'Instrumentation run failed due to 'junit.framework.AssertionFailedError''. Check device logcat for details

And here is the traceback:

10-27 19:51:29.006    4188-4188/? E/AndroidRuntime﹕ FATAL EXCEPTION: main
Process: com.example.jsonapi.sample, PID: 4188
junit.framework.AssertionFailedError
        at junit.framework.Assert.fail(Assert.java:48)
        at junit.framework.Assert.assertTrue(Assert.java:20)
        at junit.framework.Assert.assertNotNull(Assert.java:218)
        at junit.framework.Assert.assertNotNull(Assert.java:211)
        at com.example.jsonapi.JsonAPITest$2.onDone(JsonAPITest.java:101)
        at com.example.jsonapi.JsonAPITest$2.onDone(JsonAPITest.java:96)
        at org.jdeferred.impl.AbstractPromise.triggerDone(AbstractPromise.java:107)
        at org.jdeferred.impl.AbstractPromise.triggerDone(AbstractPromise.java:98)
        at org.jdeferred.impl.DeferredObject.resolve(DeferredObject.java:70)
        at com.example.jsonapi.JsonAPI$1.onResponse(JsonAPI.java:178)
        at com.example.jsonapi.JsonAPI$1.onResponse(JsonAPI.java:172)
        at com.android.volley.toolbox.StringRequest.deliverResponse(StringRequest.java:67)
        at com.android.volley.toolbox.StringRequest.deliverResponse(StringRequest.java:30)
        at com.android.volley.ExecutorDelivery$ResponseDeliveryRunnable.run(ExecutorDelivery.java:99)
        at android.os.Handler.handleCallback(Handler.java:739)
        at android.os.Handler.dispatchMessage(Handler.java:95)
        at android.os.Looper.loop(Looper.java:135)
        at android.app.ActivityThread.main(ActivityThread.java:5257)
        at java.lang.reflect.Method.invoke(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:372)
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698)

And here is the corresponding test:

public void testFindOneCustomURL() throws Exception {
    ArrayList<String> reqList = new ArrayList<>();
    reqList.add("queue.txt");

    MockWebServer server = getServerWithData(reqList);
    // Ask the server for its URL. You'll need this to make HTTP requests.
    URL baseUrl = server.getUrl("/api/v1");

    JsonAPI client = new JsonAPI(baseUrl, context);
    client.getGsonBuilder()
           .setDateFormat("yyyy-MM-dd'T'HH:mm:ss");
    client.register(new VideoQueue());
    Promise prom = client.find(new VideoQueue())
            .set("user", "212139")
            .run()
            .done(new DoneCallback<ArrayList<VideoQueue>>() {
                @Override
                public void onDone(ArrayList<VideoQueue>  videoQueue) {
                    assertNotNull(videoQueue);
                    assertNotNull(videoQueue.get(0));
                    assertNotNull(videoQueue.get(0).getId());
                    assertEquals("3310190", videoQueue.get(0).getId());
                }
            }).fail(new FailCallback() {
                @Override
                public void onFail(Object rejection) {
                    assertNull(rejection);
                }
            });

    prom.waitSafely();
    // Shut down the server. Instances cannot be reused.
    server.shutdown();

    // Make sure our client hit the server
    RecordedRequest request = server.takeRequest();
    assertEquals("/api/v1/users/212139/queue", request.getPath());
}

Answer 1

This means the assertion failed. Normally it should report that the test case fails. But if you put the assertion in a callback method, it appears to crash. Maybe this is a bug of junit.