CODEWITHSUNDEEP
pythonpytest
nonbot
nonbot

Reputation: 973

How to make a py.test failure trigger outside functions?

I am currently writing a script that installs my software-under-test then automatically runs my smoke tests using py.test. If a failure occurs during any of these tests, I would like to tell my software to not publish the software to the build servers. This is basically how it goes in pseudo-code:

def install_build_and_test():
    # some python code installs some_build
    install_my_build(some_build)

    # then I want to test my build
    subprocess.Popen(["py.test", "smoke_test_suite.py"])
    # test_failures = ???

    # If any failures occurred during testing, do not publish build 
    if test_failures is True:
        print "Build will not publish because there were errors in your logs"

    if test_failures is False:
        publish_build(some_build)

My question here is how do I use pytest failures to tell my install_and_test_build code to not publish some_build?

Upvotes: 3

Views: 800

Answers (2)

lemonhead
lemonhead

Reputation: 5518

Approach #1

This is I think the road you were heading down. Basically, just treat test.py as a black box process and use the exit code to determine if there were any test failures (e.g. if there is a non-zero exit code)

exit_code = subprocess.Popen(["py.test", "smoke_test_suite.py"]).wait()
test_failures = bool(exit_code)

Approach #2

Another even cleaner way is to run py.test in python directly.

import pytest
exit_code = pytest.main("smoke_test_suite.py")
test_failures = bool(exit_code)

Upvotes: 3

bereal
bereal

Reputation: 34282

py.test must return a non-zero exit code if the tests fail. The simplest way to handle that would be using subprocess.check_call():

try:
    subprocess.check_call(["py.test", "smoke_test_suite.py"])
except subprocess.CalledProcessError:
    print "Smoke tests have failed, not publishing"
else:
    print "Smoke tests have passed, publishing"
    # ...

Upvotes: 0

Related Questions