CODEWITHSUNDEEP
perltesting
user1836185
user1836185

Reputation: 121

Print failed subtests using Test::More and Test::Deep

I wrote a big test using Test::More and Test::Deep. The test contains subtests. When I run the code, at the last line I get:

# Looks like you failed 3 tests of 25.

How can I print a summary that specifies the subtests that failed? (Each subtest has a title.)

Upvotes: 4

Views: 1006

Answers (1)

Schwern
Schwern

Reputation: 165110

Short answer: you can't.

Slightly longer answer: you have to patch Test::Builder or TAP::Harness.

Hacky answer: put a wrapper around the subtest function which captures the name and result (subtest(), like all test functions, returns true on passing, false on failure) and outputs extra summary information in an END block. Implementation is left to the reader.

tl;dr information...

Test::Builder (and thus the whole Test ecosystem) does not report subtests in the summary. I don't believe the released version (0.9x) of Test::Builder stores the results of subtests at all. Subtests are kind of a horrible hack in Test::Builder 0.9x. Test::Builder 1.5 handles subtests much more elegantly, so if you wanted to make a patch I'd do it against 1.5.

TAP::Harness (and thus prove) does not understand subtests, as far as its concerned its garbage to be ignored. To work around this each subtest produces an extra test at the end which is all Test::Harness understands.

# TAP::Harness sees this line, the overall plan.
1..1
    # All this indented subtest info is ignored by TAP::Harness.
    ok 1
    ok 2
    not ok 3
    1..3
    # Looks like you failed 1 test of 3.
# TAP::Harness sees this line, and that all it knows about the subtest
# It doesn't even know its a subtest.
not ok 1 - foo

TAP::Harness would need to be patched to understand subtests, which would be quite welcome.

A wrapper around subtest, using Test::More's own subtest function as a guide, would look like this:

use Test::Builder;

my @Failed_Subtests;
sub my_subtest {
    my $name = shift;

    # Call subtest
    my $tb = Test::Builder->new;
    my $subtest_ok = $tb->subtest($name, @_);

    # Record the name of any failing subtests
    push @Failed_Subtests, $name if !$subtest_ok;

    return $subtest_ok;
}

# When the test is over, show the failed subtests
END {
    if( @Failed_Subtests ) {
        diag sprintf "These subtests failed: %s", join(", ", @Failed_Subtests);
    }
}

And if you use my_subtest instead of subtest you'll get something like:

# These subtests failed: bar, baz
# Looks like you failed 2 tests of 3.

Instead of defining a new subtest function, you can replace Test::More's. The body of the function would look the same, but the technique for replacing a Perl function is a subject of another question.

Upvotes: 7

Related Questions