CODEWITHSUNDEEP
.netunit-testingjenkinscode-coverage
Jeff Dege
Jeff Dege

Reputation: 11680

Are there any code coverage tools that allow me to exclude code by line?

What I want, from a .NET coverage tool, and haven't been able to find...

Ability to run unit tests, and to track which lines of code have been executed by which unit tests, and which lines of code have not been executed at all. Ability to generate interactive reports, that allow the user to view statistical summaries of coverage by assembly, file, namespace, classe, etc;, and to drill down to the source code level to see exactly what lines were and weren't covered.

Ability to run the tests and generate the reports from Jenkins, and to include the reports in the information that Jenkins displays for every build. Ability to mark source code, by line, as not needing to be covered. Ability to have the reports clearly indicate that every line of source code has either been covered, or marked as not needing to be covered.

I've been looking around at the free coverage tools, and haven't been satisfied. I recently began reading up on dotCover, and it certainly looks like it provides more of what I need. But from what I've read in the docs, it provides mechanisms for excluding code from coverage requirements by file, class, or function. But if it provides the capability to exclude code from coverage by line, I've missed it.

What I'm looking for, in essence, is a pass-fail. 90% coverage, or 80% coverage, or 95% coverage, means nothing to me. 80% coverage might mean 20% of the code doesn't need coverage, 99.9% coverage might mean that only one critical function that desperately needs testing hasn't been tested.

The way I see it there are three categories:

  1. Code that has been covered by a test
  2. Code that has been flagged as not needing to be tested
  3. Code that has not been covered by a test, and has not been flagged as not needing to be tested

And what I want, after a unit test run, is a pass, if there is no code in category 3, or a fail if there is.

And I need to be able to flag code as not needing to be tested by line, and not just by function, because when I look at a function in a coverage report, it's pretty typical for me to see certain flows of control through the function that weren't executed by the tests. Most commonly, exception or error handling. Sometimes, the effort needed to create tests that will generate the errors or exceptions needed to test these flows of control can be significant, and sometimes the likelihood of the test revealing an issue is small.

In cases like these, I would like to be able to flag the code as having been inspected, and having been determined to not need testing. And when I flag the code in this way, I want the fact that the code wasn't covered by a test to not fail the build.

Does anyone know of any NET coverage tool that does this?

Upvotes: 3

Views: 1101

Answers (1)

Shaun Wilde
Shaun Wilde

Reputation: 8358

OpenCover will allow you to track coverage by line (sequence point) and if you use the -coverbytest you can determine which test was running when the line was hit.

There is no exclude by line feature as OpenCover instruments your assembly as it is running and is not aware of the actual lines of code. You could write your own tooling to remove the results for the lines that you wish to exclude by parsing you code and then removing any sequence points from the results before you generate a report using something like ReportGenerator.

However one can argue that no code is untestable you just need to restructure it differently. Commonly this sort of code is what people say they can't test as they can't force the exception.

public void Method()
{
  try
  {
    // do something
  }
  catch (Exception ex)
  {
    // respond to exception
  }
}

but to test this all you need to do is break it down to smaller parts

internal void TryCatch<T>(Action doAction, Action<T> catchAction) where T : Exception { 
  try { doAction(); } catch (T ex) { catchAction(ex); }
}

internal void DoMethod() {
  // do something
}

internal void CatchMethod(Exception ex){
  // respond to exception
}

public void Method() {
  TryCatch<Exception>(DoMethod, CatchMethod);
}

Now each piece is individually testable without resorting to hiding any code should you so wish. e.g. in the case of TryCatch<T>() you can have tests that if the doAction throws an exception the catchException action is called or you could just exclude the method from coverage as it is so trivial.

I rarely resort to this pattern way unless I really need to test the catch block is called under certain conditions.

Upvotes: 1

Related Questions