CODEWITHSUNDEEP
pytest
user2052592
user2052592

Reputation: 11

pytest overall result 'Pass' when all tests are skipped

Currently pytest returns 0 when all tests are skipped. Is it possible to configure pytest return value to 'fail' when all tests are skipped? Or is it possible to get total number passed/failed tests in pytest at the end of execution?

Upvotes: 1

Views: 822

Answers (1)

ntki
ntki

Reputation: 2314

There is possibly a more idiomatic solution, but the best I could come up with so far is this.

Modify this example of the documentation to save the results somewhere.

# content of conftest.py
import pytest
TEST_RESULTS = []

@pytest.mark.tryfirst
def pytest_runtest_makereport(item, call, __multicall__):
    rep = __multicall__.execute()
    if rep.when == "call":
        TEST_RESULTS.append(rep.outcome)
    return rep

If you want to make the session fail on a certain condition, then you can just write a session scoped fixture-teardown to do that for you:

# conftest.py continues...
@pytest.yield_fixture(scope="session", autouse=True)
def _skipped_checker(request):
    yield
    if not [tr for tr in TEST_RESULTS if tr != "skipped"]:
        pytest.failed("All tests were skipped")

Unfortunatelly the fail (Error actually) from this will be associated to the last testcase in the session.

If you want to change the return value then you can write a hook:

# still conftest.py
def pytest_sessionfinish(session):
    if not [tr for tr in TEST_RESULTS if tr != "skipped"]:
        session.exitstatus = 10

Or just call through pytest.main() then access that variable and do you own post session checks.

import pytest
return_code = pytest.main()

import conftest
if not [tr for tr in conftest.TEST_RESULTS if tr != "skipped"]:
    sys.exit(10)
sys.exit(return_code)

Upvotes: 1

Related Questions