PowerMock test passes then fails

Question

Why are my unit-tests passing when run independently, but fail when running multiple tests?

When I execute a single unit test, my tests will successfully mock and return the expected results. However, when I run all unit-tests my previously passing test will fail.

One Test Run

shouldDoThisAgain() - Pass

Multiple Test Runs

shouldDoThis() - Pass

shouldDoThisAgain() - Fail

shouldDoThisAgainAgain() - Fail

My tests:

@PrepareForTest({OtherMethods.class})
@PowerMockIgnore("javax.management.*")
@RunWith(PowerMockRunner.class)
public class DbTest {
    @Test
    public void shouldDoThis() throws Exception() {
        Dal dalMock = mock(Dal.class)
        PowerMockito.whenNew(Dal.class).withAnyArguments().thenReturn(dalMock)
        List<Result> results = new ArrayList<Result>();
        results.add(new Result(1,2,3));
        when(dalMock.getResults()).thenReturn(results)
        assertTrue(Wrapper.MY_WRAPPER.run()); 
    }
    @Test
    public void shouldDoThisAgain() throws Exception() {
        Dal dalMock = mock(Dal.class)
        PowerMockito.whenNew(Dal.class).withAnyArguments().thenReturn(dalMock)
        List<Result> results = new ArrayList<Result>();
        results.add(new Result(2,3,4));
        when(dalMock.getResults()).thenReturn(results)
        assertTrue(Wrapper.MY_WRAPPER.run());
    }
    @Test
    public void shouldDoThisAgainAgain() throws Exception() {
        Dal dalMock = mock(Dal.class)
        PowerMockito.whenNew(Dal.class).withAnyArguments().thenReturn(dalMock)
        List<Result> results = new ArrayList<Result>();
        results.add(new Result(6,5,3));
        when(dalMock.getResults()).thenReturn(results)
        assertTrue(Wrapper.MY_WRAPPER.run());
    }
}

My classes:

public class Wrapper {
    // not Runnable
    public static final MyWrapper MY_WRAPPER = new MyWrapper(...){
        @Override
        public boolean run() {
           // returns empty list when the test is alone
           // returns 'results' variable when ran with other tests alone
           List<Result> results = OtherMethods.getDal().getResults();
           return !results.isEmpty()
        }
    };  
}

public class OtherMethods {
     private static final Logger LOGGER = LogManager.getLogger(OtherMethods.class);

     public static Dal dal;
     
     static Dal getDal() {
         if (dal == null) {
             try {   
                 dal = new Dal();
             } catch (Exception e) {
               LOGGER.fatal("DB Connection could not be created for Geonames");
               LOGGER.fatal(e);
           }
        }
        return dal;
    }
}

Answer 1

I found the solution for our project. I wrote a Logger class that calls Android's internal static Log methods. Some of my tests did not directly test the Log class. When I ignored all of them, the powermockito-based tests went green. But when these other tests were run, the powermockito-based ones would fail. Sometimes.

This approach would fail (flaky):

@RunWith(PowerMockRunner.class)
@PrepareForTest({Log.class})    // WARNING: HERE BE DRAGONS!
public class MyTest {

    @Test
    public void testMethodThatDoesNotUseStatics() {
        // ...
    }

    @Test
    public void usesStatics() {
        // ...
    }

}

Then I found out you can annotate each test method with @PrepareForTest, like this:

@RunWith(PowerMockRunner.class)
public class MyTest {

    @Test
    public void testMethodThatDoesNotUseStatics() {
        // ...
    }

    @Test
    @PrepareForTest({Log.class})    // that's the way :)
    public void usesStatics() {
        // ...
    }

}

Now the tests are green again. Yay for non-flaky tests! :)

Answer 2

Check what the behaviour of @PrepareForTest({OtherMethods.class}) is when placed at class level...

deleted in response to OP's comment
I just noticed something else:
I presume that your MyWrapper class is Runnable and as such it can only run() once, you need to reinitialise it for every test
deleted

Edit: Then your problem is in your implementation of OtherMethods class, you don't show it here and that makes it difficult for us

Answer 3

Try doing this

@PrepareForTest({OtherMethods.class})
@PowerMockIgnore("javax.management.*")
@RunWith(MowerMockRunner.class)
public class DbTest {
@Before
public void setUp(){
     Dal dalMock = mock(Dal.class);        
PowerMockito.whenNew(Dal.class).withAnyArguments().thenReturn(dalMock);
List<Result> results = new ArrayList<Result>();
    results.add(new Result(1,2,3));
    when(OtherMethods.getDal().getResults()).thenReturn(results)
}
@Test
public void shouldDoThis() throws Exception() {

    assertTrue(Wrapper.MY_WRAPPER.run());
}
@Test
public void shouldDoThisAgain() throws Exception() {

    assertTrue(Wrapper.MY_WRAPPER.run());
}
@Test
public void shouldDoThisAgainAgain() throws Exception() {

    assertTrue(Wrapper.MY_WRAPPER.run());
}
}