CODEWITHSUNDEEP
unit-testingcode-coverage
A.G.
A.G.

Reputation: 2149

Do code coverage tools necessarily have to RUN Unit tests for analysis?

I'm new to using code coverage and not up on how they behave. The tools I have tried all seem to want to run the entire suite of unit tests to do their analysis. I had though that they would somehow analyze the code statically, that is, without having to run Unit Tests.

Is my assumption incorrect?

Upvotes: 2

Views: 227

Answers (2)

Nathan Hughes
Nathan Hughes

Reputation: 96385

The usual open source tools, like Cobertura, Emma, etc., require running the tests. They use things like byte-code manipulation or AOP to instrument the code and track which lines get executed as the tests run, then produce reports of the results. The simplicity of it makes it a very attractive solution.

It's really interesting that you can approximate test coverage with static analysis, but it seems like a last-ditch option for cases where running the tests is not practical. Quoting from the abstract from the paper quoted in Ira Baxter's answer:

Test coverage is an important indicator for unit test quality. Tools such as Clover compute coverage by first instrumenting the code with logging functionality, and then logging which parts are executed during unit test runs. Since computation of test coverage is a dynamic analysis, it presupposes a working installation of the software. In the context of software quality assessment by an independent third party, a working installation is often not available. The evaluator may not have access to the required libraries or hardware platform. The installation procedure may not be automated or documented. In this paper, we propose a technique for estimating test coverage at method level through static analysis only. The technique uses slicing of static call graphs to estimate the dynamic test coverage. We explain the technique and its implementation. We validate the results of the static estimation by statistical comparison to values obtained through dynamic analysis using Clover. We found high correlation between static coverage estimation and real coverage at system level but closer analysis on package and class level reveals opportunities for further improvement.

Upvotes: 1

Ira Baxter
Ira Baxter

Reputation: 95344

Standard test coverage tools, which collect information at run time, necessarily need something to exercise the code, and test suite at least has the purpose of covering all the functionality. If you don't have a test suite, you can run ad hoc tests but your coverage isn't likely to very good, and of course its a hard experiment to repeat.

But you don't necessarily have to use dynamic analysis. This paper shows how one can get a very good approximation of test coverage by doing static analysis. I know the authors personally and have very high respect for their work.

Upvotes: 0

Related Questions