CODEWITHSUNDEEP
androiddebuggingtesting
Code-Apprentice
Code-Apprentice

Reputation: 83547

Running JUnit test multiple times gives different results

I have a test case for my app which fills in the TextViews in an Activity and then simulates clicking the Save button which commits the data to a database. I repeat this several times with different data, call Instrumentation.waitForIdleSync(), and then check that the data inserted is in fact in the database. I recently ran this test three times in a row without changing or recompiling my code. The result each time was different: one test run passed and the other two test runs reported different data items missing from the database. What could cause this kind of behavior? Is it possibly due to some race condition between competing threads? How do I debug this when the outcome differs each time I run it?

Upvotes: 2

Views: 972

Answers (3)

Code-Apprentice
Code-Apprentice

Reputation: 83547

I (finally!) found a solution to this problem. I now call finish() on the tested Activity to make sure that all of its connections to the database are closed. This seems to ensure consistency in the data when I run the assertions.

Upvotes: 1

Paul Harris
Paul Harris

Reputation: 5819

I would suggest making a probe for the database data rather than a straight assert on it. By this I mean make a piece of code that will keep checking the database for up to a certain amount of time for a condition rather than waiting for x seconds (or idle time) then check, I am not on a proper computer so the following is only pseudo code

public static void assertDatabaseHasData(String message, String dataExpected, long maxTimeToWaitFor){
    long timeToWaitUntil = System.getCurrentTimeMillis() + maxTimeToWaitFor;
    boolean expectationMatched = false;
    do {
        if(databaseCheck() == dataExpected){
            expecttionMatched == true;
        }
    }while(!expectationMatched  && System.getCurrentTimeMillis() < timeToWaituntil);
    assertTrue(message, expectationMatched);
}

When i get to a computer i will try to relook into the above and make it better (I would actually of used hamcrest rather than asserts but that is personal preference)

Upvotes: 0

JasonG
JasonG

Reputation: 5962

Looks like a race condition. remember in the world of threading there is no way to ensure runtime order.

I'm not an android dev so I'm only speculating but UI is only on one event thread generally so when you call the method from another thread (your test) you're probably breaking that as you're outside of the event thread.

You could try using a semaphore or more likely a lock on the resource. http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Lock.html http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/Semaphore.html

Upvotes: 1

Related Questions