CODEWITHSUNDEEP
pythonpytest
Isaac
Isaac

Reputation: 9652

Processing pytest test results at runtime

So I have have pytest run my tests and that's great, but I want to actually do something with the test results. I was using unittest, and that gives me a swanky results object that I can process after the tests have run. Pytest just seems to give me a big text dump - writing a parser for that sounds mind-numbingly boring.

How do I get the results into something I can use? I must be missing something.

btw - I'm running my tests using pytest.main(), not via the command line py.test. I was hoping to have some sort of result object I can interact with at runtime. I realize I can write results to disk, read from disk, parse said results and then act on results - but it seems like those disk operations are just extra steps I should be able to avoid.

Upvotes: 4

Views: 2209

Answers (2)

jxramos
jxramos

Reputation: 8266

You may be able to do some introspection with the session object

you can play around in the pytest_sessionfinish callback to inspect the session object and see what crawling actions you can do over it. I know you can get the test case names and paths and all that, but their final result status is proving elusive to me to locate in a quick time.

def pytest_sessionfinish(session):
    type(session)
    session
    import pdb
    pdb.set_trace()

(Pdb) type(session)
<class '_pytest.main.Session'>
(Pdb) session
<Session driving_develop_fw exitstatus=<ExitCode.TESTS_FAILED: 1> testsfailed=1 testscollected=2>


(Pdb) for d in dir(session): print(d)
CollectError
Failed
Interrupted
__class__
__delattr__
__dict__
__dir__
__doc__
__eq__
__format__
__ge__
__getattribute__
__gt__
__hash__
__init__
__init_subclass__
__le__
__lt__
__module__
__ne__
__new__
__reduce__
__reduce_ex__
__repr__
__setattr__
__sizeof__
__str__
__subclasshook__
__weakref__
_bestrelpathcache
_collect
_collectfile
_fixturemanager
_initialparts
_initialpaths
_matchnodes
_name2pseudofixturedef
_node_cache
_node_location_to_relpath
_nodeid
_norecursepatterns
_notfound
_parsearg
_perform_collect
_pkg_roots
_prunetraceback
_recurse
_repr_failure_py
_setupstate
_tryconvertpyarg
_visit_filter
add_marker
addfinalizer
collect
config
exitstatus
extra_keyword_matches
fspath
genitems
get_closest_marker
gethookproxy
getparent
ihook
isinitpath
items
iter_markers
iter_markers_with_node
keywords
listchain
listextrakeywords
listnames
matchnodes
name
nodeid
own_markers
parent
perform_collect
pytest_collectreport
pytest_collectstart
pytest_runtest_logreport
repr_failure
session
setup
shouldfail
shouldstop
startdir
teardown
testscollected
testsfailed
trace
warn

It is the items that are the individual test cases that were collected during the run. The session.testscollected and session.testsfailed give you a count for how many tests were collected and how many failures were seen across those tests.

Fyi the session reference seems to show up a few times down the hierarchy.

(Pdb) hex(id(session))
'0x7fb717b1dcf8'
(Pdb) hex(id(session.items[1].session))
'0x7fb717b1dcf8'

Upvotes: 1

Charlie Andrews
Charlie Andrews

Reputation: 1477

py.test result files are not really meant to be human readable. I'm not sure if there is a third-party parser for them or not. They are meant to be read by a continuous integration server like Hudson or Travis-ci. As @limelights said you get the xml files for these services with the --junitxml=path flag. More here. or with the --resultlog=path flag. More here.

Upvotes: 2

Related Questions