CODEWITHSUNDEEP
pythonpytest
Rodger Shleming
Rodger Shleming

Reputation: 165

pytest timeout for the whole session

I want to add to my PyTest program timeout option for the whole session. I have tried to use pytest-timeout, but it timeout each test separately and not the whole session together.

I Couldn't find a way to do it inside PyTest so I have tried to run PyTest with timeout command (bash) like this timeout <time> pytest --html=report. timeout wait for the process to finish and when the timeout expired it sends SIGTERM. This will kill PyTest process but will not create report (I'm using pytest-html to create report).

How can I timeout the whole PyTest process inside/outside of PyTest?

Upvotes: 2

Views: 2841

Answers (2)

Rodger Shleming
Rodger Shleming

Reputation: 165

After many tries I finally found a solution for this:

import pytest
import time
import signal

session_should_exit = False


def handle_timeout(signum, frame):
    global session_should_exit
    session_should_exit = True
    pytest.fail("Timeout!")

def pytest_sessionstart(session):
    signal.signal(signal.SIGALRM, handle_timeout)
    signal.setitimer(signal.ITIMER_REAL, 5)

def pytest_runtest_logfinish(nodeid, location):
    if session_should_exit:
        pytest.exit("Pytest exited")

The solution involve using signal to signal myself with 5 seconds (of course I should change that to parameter from the user instead). When handle_timeout executed I cannot use pytest.exit directly - Pytest capture the output of the tests, when test end it use the captured output and inject it to the log. This is why you cannot exit in the middle of the test, the captured output of this specific test will no be logged. Instead I save flag that I should exit and fail the test. On the hook pytest_runtest_logfinish I finally can exit (This hook executed after the captured output injected to the logs

Upvotes: 3

Valentin M
Valentin M

Reputation: 297

A fairly simple way to implement quickly this could be to create an autouse fixture that checks current timestamp against a global timestamp.

Here is a simplistic example to illustrate the idea:

import pytest
from time import time, sleep

pytest.global_timeout = 5


@pytest.fixture(autouse=True)
def global_timeout():
    if not hasattr(pytest, "start"):
        pytest.start = time()

    if time() - pytest.start >= pytest.global_timeout:
        pytest.exit("## Timeout exceeded")


def test_zero():
    assert 3 == 5


def test_one():
    pass


def test_two():
    sleep(8)
    pass


def test_three():
    pass

Execution:

$ pytest
==================================================== test session starts ====================================================
platform linux2 -- Python 2.7.15+, ...
 ...
collected 4 items                                                                                                           

test_my_feature.py::test_zero FAILED                                                                                  [ 25%]
test_my_feature.py::test_one PASSED                                                                                   [ 50%]
test_my_feature.py::test_two PASSED                                                                                   [ 75%]
test_my_feature.py::test_three 

========================================================= FAILURES ==========================================================
_________________________________________________________ test_zero _________________________________________________________

    def test_zero():
>       assert 3 == 5
E       assert 3 == 5
E         -3
E         +5

test_my_feature.py:17: AssertionError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Exit: ## Timeout exceeded !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================ 1 failed, 2 passed in 8.28 seconds =============================================```

Upvotes: -1

Related Questions