Coverage report for Perl modules by cpancover.com

Question

cpancover.com produces reports on the code coverage of Perl modules.

I understand it's just a webserver using Devel::Cover, but I would like to simply understand the meaning of reports like: http://cpancover.com/latest/JSON-PP-4.02/blib-lib-JSON-PP-pm--condition.html

where a table has columns like:

• !l
• l&&!r
• l&&r

Can you, please, link a relevant document to understand this or provide me some initial guidance to these reports?

Answer 1

Here is one part of documentation that provides information on the report format.

In coverage testing, as Devel::Cover tells us, there may be places in code which cannot be checked: they are "uncoverable"

Sometimes you have code which is uncoverable for some reason. Perhaps it is an else clause that cannot be reached, or a check for an error condition that should never happen. You can tell Devel::Cover that certain criteria are uncoverable and then they are not counted as errors when they are not exercised. In fact, they are counted as errors if they are exercised. [...]

In dealing with this one can consider statements, branches, conditions, and subroutines. The example report you provide deals with Conditions, and for that the docs say

Because of the way in which Perl short-circuits boolean operations, there are three ways in which such conditionals can be uncoverable. In the case of $x && $y for example, the left operator may never be true, the right operator may never be true, and the whole operation may never be false. These conditions may be modelled thus:

# uncoverable branch true
# uncoverable condition left
# uncoverable condition false
if ($x && !$y) {
    $x++;  # uncoverable statement
}

# uncoverable branch true
# uncoverable condition right
# uncoverable condition false
if (!$x && $y) {
}

With this discussion the linked report makes more sense.

Here is how I'd think of the first line in your link, for example. It is a A && B kind of condition and it has the following coverage of the possible cases:

• left side false --- covered in tests (two)

• left true but right false --- not covered

• both left and right true --- covered (two)

Then we click on "line 214" (it's a link) and see the overall percentile coverage (66 -- two thirds). The color is telling and the codes can be seen on this page

The docs also say that there is a variety of report formats so you may want to dig further. However, I don't readily see where to look, what I find a little unsettling.

Answer 2

l and r refer to the right-hand side and left-hand side of the expression, and the number represents the number of times the associated expression was true.

For example, there is a coverage of 0 for l && !r for the following expression:

exists $self->{'true'} and exists $self->{'false'} 

That means that 0 tests cover the situation where the following is true:

(exists $self->{'true'}) && !(exists $self->{'false'})

---

Logical-and and logical-or have two inputs that can take each of two values.

+-----------------+-----------------+
| Left-hand side  | Right-hand side |
+-----------------+-----------------+
| False           | False           |
| False           | True            |
| True            | False           |
| True            | True            |
+-----------------+-----------------+

However, because of short-circuiting, Perl doesn't always evaluate the right-hand side. The cases are really the following:

+-----------------------------------+
| Logical AND                       |
+-----------------+-----------------+
| Left-hand side  | Right-hand side |
+-----------------+-----------------+
| False           | -               |
| True            | False           |
| True            | True            |
+-----------------+-----------------+

+-----------------------------------+
| Logical OR                        |
+-----------------+-----------------+
| Left-hand side  | Right-hand side |
+-----------------+-----------------+
| False           | False           |
| False           | True            |
| True            | -               |
+-----------------+-----------------+

Devel::Cover is reporting which of these sets of inputs have been tested.

+--------------------------------------------------------+
| Logical AND                                            |
+--------------------+-----------------+-----------------+
| Devel::Cover label | Left-hand side  | Right-hand side |
+--------------------+-----------------+-----------------+
| !l                 | False           | -               |
| l && !r            | True            | False           |
| l && r             | True            | True            |
+--------------------+-----------------+-----------------+

+--------------------------------------------------------+
| Logical OR                                             |
+--------------------+-----------------+-----------------+
| Devel::Cover label | Left-hand side  | Right-hand side |
+--------------------+-----------------+-----------------+
| l                  | True            | -               |
| !l && r            | False           | True            |
| !l && !r           | False           | False           |
+--------------------+-----------------+-----------------+

If we look at the first line of the linked page, we see

!l        2
l && !r   0
l && r    2

for

exists $self->{'true'} and exists $self->{'false'} 

That means

!l       meaning  !(exists $self->{'true'})                               was true  2  times.
l && !r  meaning  (exists $self->{'true'}) && !(exists $self->{'false'})  was true  0  times.
l && r   meaning  (exists $self->{'true'}) && (exists $self->{'false'})   was true  2  times.

This means that (exists $self->{'true'}) && !(exists $self->{'false'}) being true was never tested.