CODEWITHSUNDEEP
pythonunit-testingpytest
benevolentprof
benevolentprof

Reputation: 1527

Ordering of py.test functions results in failure

I'm teaching an introduction to programming course using Python. I am using py.test to grade student work. I have a situation where the py.test reports a failed test case when the test functions are in one order, but reports passing when the test functions are in another order.

I've created a project on Github with the failing program and test code. But here's an excerpt from the test program.

from exercise3 import union, intersection, difference

###########
# TABLES ##
###########
GRADUATES = [["Number", "Surname", "Age"],
             [7274, "Robinson", 37],
             [7432, "O'Malley", 39],
             [9824, "Darkes", 38]]

MANAGERS = [["Number", "Surname", "Age"],
            [9297, "O'Malley", 56],
            [7432, "O'Malley", 39],
            [9824, "Darkes", 38]]


#####################
# HELPER FUNCTIONS ##
#####################
def is_equal(t1, t2):
    return sorted(t1) == sorted(t2)


###################
# TEST FUNCTIONS ##
###################


def test_union():
    """
    Test union operation.
    """

    result = [["Number", "Surname", "Age"],
              [7274, "Robinson", 37],
              [9297, "O'Malley", 56],
              [7432, "O'Malley", 39],
              [9824, "Darkes", 38]]

    assert is_equal(result, union(GRADUATES, MANAGERS))


def test_intersection():
    """
    Test intersection operation.
    """
    result = [["Number", "Surname", "Age"],
              [7432, "O'Malley", 39],
              [9824, "Darkes", 38]]

    assert is_equal(intersection(GRADUATES, MANAGERS), result)

If I put the test_union function first, and the test_intersection function second, the latter fails. Here is the output from py.test. It looks like the result variable in test_intersection is using the value from the test_union function.

/Users/ses/anaconda/bin/python "/Applications/PyCharm Educational.app/Contents/helpers/pycharm/pytestrunner.py" -p pytest_teamcity /Users/ses/PycharmProjects/pytest_weirdness/test_exercise3.py
Testing started at 1:11 PM ...
============================= test session starts ==============================
platform darwin -- Python 2.7.10 -- py-1.4.27 -- pytest-2.7.1
rootdir: /Users/ses/PycharmProjects/pytest_weirdness, inifile: 
collected 3 items

../../../../Users/ses/PycharmProjects/pytest_weirdness/test_exercise3.py .F
def test_intersection():
        """
        Test intersection operation.
        """
        result = [["Number", "Surname", "Age"],
                  [7432, "O'Malley", 39],
                  [9824, "Darkes", 38]]

    >       assert is_equal(intersection(GRADUATES, MANAGERS), result)
    E       assert is_equal([['Number', 'Surname', 'Age'], [7432, "O'Malley", 39], [9824, 'Darkes', 38], [9297, "O'Malley", 56]], [['Number', 'Surname', 'Age'], [7432, "O'Malley", 39], [9824, 'Darkes', 38]])
    E        +  where [['Number', 'Surname', 'Age'], [7432, "O'Malley", 39], [9824, 'Darkes', 38], [9297, "O'Malley", 56]] = intersection([['Number', 'Surname', 'Age'], [7274, 'Robinson', 37], [7432, "O'Malley", 39], [9824, 'Darkes', 38], [9297, "O'Malley", 56]], [['Number', 'Surname', 'Age'], [9297, "O'Malley", 56], [7432, "O'Malley", 39], [9824, 'Darkes', 38]])

If I reverse the order, both tests pass. If I run one test at a time, the test case passes.

This is very confusing to me because I thought the order of test cases didn't matter. Also, I don't have the problem with my own code. The assignment isn't due until tonight, so I'm not sharing the solution yet, but the original repo is also available on GitHub.

I'm using Anaconda 2.3.0 distribution and PyCharm Educational Edition with Python 2.7.10 and pytest-2.7.1

Upvotes: 1

Views: 700

Answers (1)

memoselyk
memoselyk

Reputation: 4128

The problem is that you are passing a list, which is mutable, but functions under test have side-effect on them.

One possible solution would be to pass a deepcopy of the list to the tested function. Using copy module from standard library.

E.g.

assert is_equal(result, union(copy.deepcopy(GRADUATES), copy.deepcopy(MANAGERS)))

Upvotes: 2

Related Questions