CODEWITHSUNDEEP
pythonunit-testingdiscoverypython-coverage
Helmut Grohne
Helmut Grohne

Reputation: 6778

Python test discovery with doctests, coverage and parallelism

... and a pony! No, seriously. I am looking for a way to organize tests that "just works". Most things do work, but not all pieces fit together. So here is what I want:

My current approach involves a tests directory and the load_tests protocol. All files contained are named like test_*.py. This makes python -m unittest discover just work, if I create a file test_doctests.py with the following content.

import doctest
import mymodule1, mymodule2
def load_tests(loader, tests, ignore):
    tests.addTests(doctest.DocTestSuite(mymodule1))
    tests.addTests(doctest.DocTestSuite(mymodule2))
    return tests

This approach also has the upside that one can use setuptools and supply setup(test_suite="unittest2.collector").

However this approach has a few problems.

How can I make things work better than this or fix some of the issues above?

Upvotes: 10

Views: 2162

Answers (2)

ajk8
ajk8

Reputation: 221

This is an old question, but the problem still persists for some of us! I was just working through it and found a solution similar to kaapstorm's, but with much nicer output. I use py.test to run it, but I think it should be compatible across test runners:

import doctest
from mypackage import mymodule

def test_doctest():
    results = doctest.testmod(mymodule)
    if results.failed:
        raise Exception(results)

What I end up with in a failure case is the printed stdout output that you would get from running doctest manually, with an additional exception that looks like this:

Exception: TestResults(failed=1, attempted=21)

As kaapstrom mentioned, it doesn't count tests properly (unless there are failures) but I find that isn't worth a whole lot provided the coverage numbers come back high :)

Upvotes: 2

kaapstorm
kaapstorm

Reputation: 1091

I use nose, and found your question when I experienced the same problem.

What I've ended up going with is not pretty, but it does run the tests. 

import doctest
import mymodule1, mymodule2

def test_mymodule1():
    assert doctest.testmod(mymodule1, raise_on_error=True)

def test_mymodule2():
    assert doctest.testmod(mymodule2, raise_on_error=True)

Unfortunately it runs all the doctests in a module as a single test. But if things go wrong, at least I know where to start looking. A failure results in a DocTestFailure, with a useful message:

DocTestFailure: <DocTest mymodule1.myfunc from /path/to/mymodule1.py:63 (4 examples)>

Upvotes: 1

Related Questions