CODEWITHSUNDEEP
testingcode-coveragelogical-operators
John V
John V

Reputation: 5047

Issue with examples of decision and statement coverage disadvantage

I'm studying the difference between these and cannot understand the difference. It seems the same to me. Arent they the same? The risk is coming from short-circuiting, isnt it?

Statement coverage does not call for testing logical operators. In C++ and C these operators are &&, ||, and ?:. Statement coverage cannot distinguish the code separated by logical operators from the rest of the statement. Executing any part of the code in a statement causes statement coverage to declare the whole statement fully covered. When logical operators avoid unnecessary evaluation (by short circuit), statement coverage gives an inflated coverage measurement.

void function(const char* string1, const char* string2 = NULL);
...
void function(const char* string1, const char* string2)
{
    if (condition || strcmp(string1, string2) == 0) // Oops, possible null pointer passed to strcmp
    ...
}

Decision coverage - A disadvantage is that this metric ignores branches within Boolean expressions which occur due to short-circuit operators. For example, consider the following C/C++/Java code fragment:

if (condition1 && (condition2 || function1()))
    statement1;
else
    statement2;

Upvotes: 0

Views: 2385

Answers (2)

Edu
Edu

Reputation: 2052

As far as I see, the risk in those examples is coming from the short-circuiting and they are indeed kind of equivalent. In some other case there could be a difference. For example consider the following case:

if (a>5)
    b = 6
if (a<20)
    c = 4 + b

For statement coverage it would be enough to test with a=10 (all lines will get executed) and the test would pass.

For decision coverage, it would be necessary to evaluate all the if statements to both true and false. The 100% decision coverage can be achieved for example with two tests a=30 and a=0. The latter will fail as variable bhas not been set in that case.

So, decision coverage is more powerful; actually 100% decision coverage guarantees 100% statement coverage, but not vice versa. However, it is still possbile to construct examples where decision coverage is not enough to expose bugs, regardless whether the language has short-circuiting.

Let's consider a situation where there wouldn't be short-circuiting

if (a>0 || b>0)
    c = 5 / b

Let's test this with input data a=0; b=1; a=0, b=0. The first one is enough to provide statement coverage. Both together provide decision coverage. But neither exposes the division by zero inside the if.

Upvotes: 1

Shaun Wilde
Shaun Wilde

Reputation: 8358

They are slightly different but both require short-circuit operations to demonstrate the problem as you correctly surmised.

Problem with Statement Coverage

if (complex-condition-evaluation-with-short-circuits) 
  doAction();

Statement coverage may record that you visited both of the above lines but you may never know if you executed a scenario where only parts of the complex condition were evaluated.

i.e. in your example was strcmp(...) ever executed?

Issue with Branch Coverage

if (complex-condition-evaluation-with-short-circuits) 
  doAction(); 
else
  doOtherAction();

Branch coverage may record that you pass through all paths in the (source) code but will ignore the branches that are part of the complex condition evaluation (as stated).

i.e. in your example branch coverage does not consider the || as an alternate branch and sees it as only two paths through the code when in fact it is four.

path1 (calls doAction) => condition1 == true, condition2 == true, function1() == ?
path2 (calls doAction) => condition1 == true, condition2 == false, function1() == true
path3 (calls doOtherAction) => condition1 == true, condition2 == false, function1() == false
path4 (calls doOtherAction) => condition1 == false, condition2 == ?, function1() == ?

I hope this helps

Upvotes: 1

Related Questions