CODEWITHSUNDEEP
pythonpytest
Yoel Einhoren
Yoel Einhoren

Reputation: 41

pytest: Mark a test as a "must pass" and stop testing if it fails

I am trying to implement a new pytest marker, called @pytest.mark.must_pass, to indicate that if the marked test fails, pytest should skip all subsequent tests and terminate.

I have been able to use the pytest_runtest_call hook to get pytest to terminate if the marked test failed, but I am using pytest.exit, which does not print a traceback, nor does it show the failure indication for the test in question.

I need this failure to appear as any other test failure, except that pytest stops testing after it prints whatever it needs to print to detail the failure.

My code so far:

# Copied this implementation from _pytest.runner
def pytest_runtest_call(item):
    _update_current_test_var(item, "call")
    try:
        del sys.last_type
        del sys.last_value
        del sys.last_traceback
    except AttributeError:
        pass
    try:
        item.runtest()
    except Exception:
        # Store trace info to allow postmortem debugging
        type, value, tb = sys.exc_info()
        assert tb is not None
        tb = tb.tb_next  # Skip *this* frame
        sys.last_type = type
        sys.last_value = value
        sys.last_traceback = tb
        del type, value, tb  # Get rid of these in this frame

        # If test is marked as must pass, stop testing here
        if item.iter_markers(name = "must_pass"):
            pytest.exit('Test marked as "must_pass" failed, terminating.')

        raise

Is there already a mechanism for doing this built into pytest?

Any help will be greatly appreciated.

Thanks.

Upvotes: 3

Views: 3141

Answers (2)

user2394284
user2394284

Reputation: 6018

Pasting the code in the accepted answer yields this:

'must_pass' not found in `markers` configuration option

For anyone coming here wanting to use – not implement – this, the same can be achieved with pytest-dependency:

  1. pip install pytest-dependency
  2. See this answer for usage.

Upvotes: 0

gold_cy
gold_cy

Reputation: 14216

So this can be achieved by using pytest_runtest_makereport and pytest_runtest_setup

In your conftest.py you would place the following:

import pytest


def pytest_runtest_makereport(item, call):
    if item.iter_markers(name='must_pass'):
        if call.excinfo is not None:
            parent = item.parent
            parent._mpfailed = item


def pytest_runtest_setup(item):
    must_pass_failed = getattr(item.parent, '_mpfailed', None)
    if must_pass_failed is not None:
        pytest.skip('must pass test failed (%s)' % must_pass_failed.name)

And now when we test it with the following:

import pytest


def foo(a, b):
    return a + b


def test_foo_1():
    assert foo(1, 1) == 2


@pytest.mark.must_pass
def test_foo_2():
    assert foo(2, 2) == 6


def test_foo_3():
    assert foo(3, 3) == 6


def test_foo_4():
    assert foo(4, 4) == 8

We see the desired output:

     ▲ = pytest test.py 
=============================================================== test session starts ================================================================
platform darwin -- Python 3.6.5, pytest-4.6.2, py-1.8.0, pluggy-0.12.0
rootdir: /Users/foo/Desktop/testing, inifile: pytest.ini
plugins: cov-2.7.1
collected 4 items                                                                                                                                  

test.py .Fss                                                                                                                                 [100%]

===================================================================== FAILURES =====================================================================
____________________________________________________________________ test_foo_2 ____________________________________________________________________

    @pytest.mark.must_pass
    def test_foo_2():
>       assert foo(2, 2) == 6
E       assert 4 == 6
E        +  where 4 = foo(2, 2)

test.py:14: AssertionError
================================================== 1 failed, 1 passed, 2 skipped in 0.08 seconds ===================================================

Upvotes: 5

Related Questions