CODEWITHSUNDEEP
pythonpytest
George
George

Reputation: 5681

I want a failed test to be OK

I have a test which fails, but I'm expecting it to fail and I want pytest to say that it is passes. How can I do that?

For example, I have a minimum value = 30 and a maximum = 40.

Here's what I'm doing:

@pytest.mark.parametrize(
        "minimum, maximum, expected_min, expected_max", [
            (13, 15, 34, 45),
            (30, 40, 30, 40),
            ("sd", 3, 34, 45),
        ])

I am receiving:

How can I have a report that says that all tests passed?

Upvotes: 2

Views: 675

Answers (2)

Tagc
Tagc

Reputation: 9076

To build on frollo's answer with a practical example based on your use case, consider the code below. We split the original set of test data into two groups: those that we expect to pass and those that we expect to fail. Unit tests like these should be testing one kind of situation in isolation, making each test case easier to write and easier to understand.

import pytest


class TestMinMax:
    @pytest.mark.parametrize(
        "minimum, maximum, expected_min, expected_max", [
            (30, 40, 30, 40),
        ])
    def test_valid_examples(self, minimum, maximum, expected_min, expected_max):
        assert minimum == expected_min
        assert maximum == expected_max

    @pytest.mark.parametrize(
        "minimum, maximum, expected_min, expected_max", [
            (13, 15, 34, 45),
            ("sd", 3, 34, 45),
        ])
    def test_invalid_examples(self, minimum, maximum, expected_min, expected_max):
        with pytest.raises(AssertionError):
            assert minimum == expected_min
            assert maximum == expected_max


if __name__ == '__main__':
    pytest.main(args=[__file__])

Output

============================= test session starts =============================
platform win32 -- Python 3.5.2, pytest-3.0.1, py-1.4.31, pluggy-0.3.1
rootdir: C:\Users\<<user>>\.PyCharmCE2016.3\config\scratches, inifile: 
collected 3 items

scratch_3.py ...

========================== 3 passed in 0.02 seconds ===========================

Upvotes: 3

frollo
frollo

Reputation: 1374

The best course of action (if this is the intended behaviour for your code) is to edit the test to reflect the intended behaviour. Test cases are not simply used to get reports you can show your manager/customer/colleague/etc, they are also a form of documentation. Test cases show people the expected behaviour of your code, so anyone who sees that test is going to assume your code will accept all the inputs shown there.

If the first and third input are illegal ones you should create a different test case, which tests how the code deals with illegal input.

Upvotes: 4

Related Questions