CODEWITHSUNDEEP
pythontesting
Rabarberski
Rabarberski

Reputation: 24922

Good strategy to perform unit tests on program that prints to stdout?

I have a python program of about 500 lines that writes to stdout (using print statements). Now I'd like to make some changes and do refactoring of the program, but I want to make sure that in the process of doing so I keep getting the same output (given the same input, of course).

What would be a good strategy to do so, without rewriting the functions to return strings (allowing for easier testing) instead of the current print-ing?

I though of redirecting the initial output (before I start changing it) to a text file. How can I then easily and automatically check the output of the modified program with the textfile (without redirecting that output again to a temporary text file and comparing the files)?

Edit: This is the solution I settled on:

def test_something():
    # using lambda because we might test function with parameters
    f = lambda: thing_to_test
    test_generic('expect_output.txt', f)

def test_generic(filename_expected, function_to_test):

    #Run and write to tmp file
    tmpfile = 'test-tmp.txt'
    sys.stdout = open(tmpfile, 'w')
    function_to_test()
    sys.stdout = sys.__stdout__

    #compare with expected output
    expected = open(filename_expected).read()
    result = open(tmpfile).read()
    d = difflib.Differ()
    diff = d.compare(expected.splitlines(), result.splitlines())

    #print result (different lines only)
    diff_lines_only = [line for line in diff if line[0] in "+-?"]
    if not diff_lines_only:
        print "Test succeeded (%s)\n" % filename_expected
        os.remove(tmpfile)
    else:
        print "Test FAILED (%s):\n" % filename_expected
        print '\n'.join(list(diff_lines))

Edit 2: Actually, I think the doctest solution which I provided as an answer below is much nicer.

Upvotes: 3

Views: 296

Answers (4)

Rabarberski
Rabarberski

Reputation: 24922

A solution using the doctest library, which I consider actually much nicer and self-contained as the test code and the expected output are now together in one file.

In a python script (actually, I included it in my main program, and is executed if I provide "test" as a first command line argument):

import doctest
doctest.testfile('test-program.txt', optionflags = doctest.NORMALIZE_WHITESPACE)

And the test file test-program.txt now goes along the lines of:

>>> import my_python_script
>>> whatever_I want_to_test_or_call_goes_here
and_here_I pasted_the_expected_output

This has the added benefit of having access to all doctest features (such as the -v switch for more verbose output). So I just do the following from the command line to get a full report:

 C:\wherever> python my_python_script test -v

Upvotes: 1

hpk42
hpk42

Reputation: 23571

Interesting little example question - i took it to create a print/capture/diffcompare solution using pytest. Note that this example makes advanced use of pytest features but those are linked and fully documented and are useful also in many other situations. Effectively, the solution needs less than half the code of the current top solution and you may find it nice to be able to selectively run tests or some of the other other pytest features.

Upvotes: 1

Bakuriu
Bakuriu

Reputation: 101959

You can obtain the string representing the output of your program redirecting sys.stdout. To compare the outputs you can use the difflib module. In particular the Differ class does more or less what the diff command does:

>>> import difflib
>>> text = '''bar
... baz
... '''
>>> text2 = '''foo
... bar
... '''
>>> d = difflib.Differ()
>>> for line in d.compare(text.splitlines(), text2.splitlines()):
...     print line
... 
+ foo
  bar
- baz

If I'm not mistaken unittest2's assertEqual already tries to show the difference of the strings, but I do not know at what level and if the output is simple enough.

Upvotes: 2

Brian Campbell
Brian Campbell

Reputation: 332846

Assuming you're running Bash, you could run diff -u orig-file <(python my-program.py). This will do a diff between the original file (which you've already written your original output to), and a named pipe that your program will write out to.

Here's a quick trivial example, using echo instead of an actual Python script:

$ diff -u <(echo $'foo\nbar\nbaz') <(echo $'foo\nbar\nquux')
--- /dev/fd/63  2012-11-08 15:07:09.000000000 -0500
+++ /dev/fd/62  2012-11-08 15:07:09.000000000 -0500
@@ -1,3 +1,3 @@
 foo
 bar
-baz
+quux

Upvotes: 1

Related Questions