CODEWITHSUNDEEP
javajunit
Jakemangan1
Jakemangan1

Reputation: 21

JUnit tests succeed individually, but when executed together fail and cause errors?

I have three JUnit tests, which are shown below. These test all succeed if they are executed individually i.e. commenting out the two other tests and only executing one.

However, if I execute all three tests uncommented then "testOrderDatabaseReturnsOrdersCorrectly" produces an error and "testOrderDatabaseRemovesOrdersCorrectly" fails.

I really don't understand why this is happening. I'm using @Before to set up before each test, so the conditions for all three tests should be the same? Why are some of them failing when they work fine individually?

@Before
public void setup()
{
    sys = new OrderSystem();
    sys.getDb().clearDb();
}


@Test
public void testOrderDatabaseAddsOrders()
{
    sys.getDb().clearDb();
    sys.createOrder(25);
    assertEquals(sys.getDb().getDbArrayList().size(), 1);

    sys.createOrder(30);
    assertEquals(sys.getDb().getDbArrayList().size(), 2);

    sys.createOrder(35);
    assertEquals(sys.getDb().getDbArrayList().size(), 3);
}


@Test
public void testOrderDatabaseRemovesOrdersCorrectly()
{
    sys.createOrder(25);
    assertEquals(sys.getDb().getDbArrayList().size(), 1);
    sys.removeOrder("BRICK1");
    assertEquals(sys.getDb().getDbArrayList().size(), 0);
}

@Test
public void testOrderDatabaseReturnsOrdersCorrectly()
{
    System.out.println("Size of db: " + sys.getDb().getDbArrayList().size());
    sys.createOrder(25);
    System.out.println("Size of db: " + sys.getDb().getDbArrayList().size());
    BrickOrder o = sys.getOrder("BRICK1");
    assertEquals(o.getNumberOfBricks(), 25);
}

Upvotes: 2

Views: 1419

Answers (3)

Sanjay Bharwani
Sanjay Bharwani

Reputation: 4829

I had similar problem, so sharing as it will help some one.

The test which was giving me tough time was using OutputCaptureExtension and using this I was trying to verify the logging as part of Unit test

@ExtendWith({MockitoExtension.class, OutputCaptureExtension.class})
class SaveHistoryService {

and my test looked like below snippet

@Test
    void saveHistory(final CapturedOutput output) {
assertThat(output.getOut()).contains(format("Record saved successfully"));
}

Now this test works fine when ran individually however when on CI it runs the entire stack it was failing continously.

That clearly explains there is some dependency between the tests.

In my case the problem was, the profiles set in previous tests.

We used info level log only in non-prod profiles by configuring in logback-spring.xml as below

<springProfile name="non-prod">
        <logger name="com.demo" level="INFO"/>
        <root level="INFO">
            <appender-ref ref="CONSOLE" />
        </root>
    </springProfile>


    <springProfile name="prod">
        <logger name="com.demo" level="WARN"/>
        <root level="WARN">
            <appender-ref ref="CONSOLE" />
        </root>
    </springProfile>

Now the test which was running before the failing test was using

@ActiveProfiles("test")
@RestClientTest(MyRestClient.class)
public class MyRestClientTest {

and post completion of the test, for the next test the app is still in test profile which means no info level logs

So, the solution is to amend the previous test to use

@ActiveProfiles({"test","non-prod"})

Upvotes: 0

Gray
Gray

Reputation: 116918

However, if I execute all three tests uncommented then "testOrderDatabaseReturnsOrdersCorrectly" produces an error and "testOrderDatabaseRemovesOrdersCorrectly" fails.

Your problem most likely is that the results from one of the test methods is not being cleared when the next test method is run and they are clashing. Maybe you are using a memory database like H2 which is not being fully cleared even though you are calling sys.getDb().clearDb();?

Couple of ways you can verify this:

  • First put a System.out.println() message in setup() to make sure it is being called before each method.
  • At the start of your test methods, do a lookup in your database to see if there are results there to verify the clearDb() did something. I suspect you will see that it isn't working fully.
  • Change the test methods to use non overlapping tables or to use non overlapping data to see if that works. For example, create a int orderNumber field and do a ++ on it in each test method to make sure you are using a new order. This is a work around of course. It would be better to understand why clear isn't doing what you want.

How to fix this is a more complicated problem that depends on what actually is backing your db. Maybe you just have some bugs in your clearDb() method. I mention H2 because even if you setup a brand new database connection, the old one never is destroyed and will be reused. If it is a SQL database, fully dropping the tables and recreating them is one thing that forces even a persistent memory database like H2 to clear its stuff.

Hope this helps.

Upvotes: 1

Jakemangan1
Jakemangan1

Reputation: 21

I have figured out what was causing the errors and strange test behaviour.

I had a field variable which was declared static. For some reason this was not being reset before each test, even though the sys variable was being reset each time. When I removed the static declaration for this variable all of the tests succeeded.

Upvotes: 0

Related Questions