CODEWITHSUNDEEP
unit-testingcode-coverage
nbari
nbari

Reputation: 26895

100% code coverage didn't cover all my real cases

I created an app, did some unit tests and got close to the 100% code coverage.

While testing the application in different conditions, configurations, etc. The app failed more than once, therefore I created/increased the current tests to cover those possibilities, but probably I still missing some cases, that the 100% coverage don't reveal.

Because of this I would like to know if there is a methodology or technique that could complement or help to cover all the possible permutations/cases and based on that create better unit tests and code coverage.

Upvotes: 2

Views: 1285

Answers (2)

Asaph
Asaph

Reputation: 162761

Increasing code coverage can increase your confidence in the correctness of the code but even when all code paths are covered by tests, it's often impractical to cover all possible inputs too. As an example, take this this method:

public static int divide(int numerator, int denominator) {
    return numerator / denominator;
}

You could trivially write a unit test that covered 100% of the lines of this method:

@Test
public void testDivide() {
    assertEquals(2, MathHelper.divide(4, 2));
}

But you would still have a divide by zero error crop up in the case of denominator = 0.

The best you can do is try to think of as many interesting edge case inputs as you reasonably can and write tests around them. And when you do encounter bugs with novel inputs, fix the bugs and write new tests (as you have done) to prevent regressions.

It's also often helpful to consider what might go wrong with libraries and/or external dependencies of your code, which have many code paths of their own that are not showing up in code coverage metrics measuring just your codebase. What happens when the database is unavailable? Network unavailable? Disk full? etc.

For some useful tips on how to get better at thinking of interesting edge cases see this Quora question.

Upvotes: 4

kidshaw
kidshaw

Reputation: 3451

100% code coverage proves all your code executes within a unit test. It doesn't mean things can't go wrong.

It also doesn't prove you've got good code. If for example, you were using a class which accesses a file, you can mock and test all the expected scenarios but good practise would be to still have something to catch any exception thrown in unexpected scenarios - that creates untestable code blocks, how do you test for the unexpected?

Whilst not a bad thing, a more realistic coverage around 80% but with multiple scenario based tests will yield better quality in my opinion.

Personally, unit tests are as much about locking in functionality as proving it works. I'd sooner have 50 tests overlapping the same core code than proving every property with a backing store.

Unfortunately, I don't know of any complementary techniques or metrics other than diligent and competent developing.

Upvotes: 1

Related Questions