CODEWITHSUNDEEP
unit-testinglogging
Coderer
Coderer

Reputation: 27304

Should I change the log level of my unit test if it's supposed to produce an error?

I have a unit test that creates an error condition. Normally, the class under test writes this error to the logs (using log4j in this case, but I don't think that matters). I can change the log level temporarily, using

Logger targetLogger = Logger.getLogger(ClassUnderTest.class);
Level oldLvl = targetLogger.getLevel();
targetLogger.setLevel(Level.FATAL);

theTestObject.doABadThing();

assertTrue(theTestObject.hadAnError());

targetLogger.setLevel(oldLvl);

but that also means that if an unrelated / unintended error occurs during testing, I won't see that information in the logs either.

Is there a best practice or common pattern I'm supposed to use here? I don't like prodding the log levels if I can help it, but I also don't like having a bunch of ERROR noise in the test output, which could scare future developers.

Upvotes: 2

Views: 1146

Answers (2)

Woot4Moo
Woot4Moo

Reputation: 24336

The way I approach this assuming an XUnit style of unit testing (Junit,Pyunit, etc) 

@Test(expected = MyException)  
 foo_1() throws Exception
{
    theTestObject.doABadThing(); //MyException here
}

The issue with doing logging is that someone needs to go and actually parse the log file, this is time consuming and error prone. However the test will pass above if MyException is generated and fail if it isn't. This in turn allows you to fail the build automatically instead of hoping the tester read the logs correctly.

Upvotes: 0

thiton
thiton

Reputation: 36059

If your logging layer permits, it is a good practice to make an assertion on the error message. You can do it by implementing your own logger that just asserts on the message (without output), or by using a memory-buffer logger and then check on the contents of the log buffer.

Under no circumstances should the error message end up in the unit-test execution log. This will cause people to get used to errors in the log and mask other errors. In short, your options are:

  1. Most preferred: Catch the message in the harness and assert on it.
  2. Somewhat OK: Raise the level and ignore the message.
  3. Not OK: Don't do anything and let the log message reach stderr/syslog.

Upvotes: 1

Related Questions