AttributeError: 'Config' object has no attribute 'test_system'

Question

rootdir: python, inifile: pytest.ini, testpaths: test_cases
plugins: metadata-1.8.0, html-1.22.0, sugar-0.9.2, timeout-1.3.3, forked-1.0.2, xdist-1.29.0, repeat-0.8.0
timeout: 3600.0s
timeout method: signal
timeout func_only: False
gw0 ok / gw1 ok / gw2 ok
../python-venv/lib/python2.7/site-packages/pluggy/hooks.py:289: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
../python-venv/lib/python2.7/site-packages/pluggy/manager.py:87: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../python-venv/lib/python2.7/site-packages/pluggy/manager.py:81: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
../python-venv/lib/python2.7/site-packages/_pytest/python.py:234: in pytest_pycollect_makeitem
    res = list(collector._genfunctions(name, obj))
../python-venv/lib/python2.7/site-packages/_pytest/python.py:410: in _genfunctions
    self.ihook.pytest_generate_tests(metafunc=metafunc)
../python-venv/lib/python2.7/site-packages/pluggy/hooks.py:289: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
../python-venv/lib/python2.7/site-packages/pluggy/manager.py:87: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../python-venv/lib/python2.7/site-packages/pluggy/manager.py:81: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
../../modules/test/python/test_cases/uts_plugin.py:20: in pytest_generate_tests
    for build_config in metafunc.config.test_system.build_configuration.get(
E   AttributeError: 'Config' object has no attribute 'test_system'
gw0 [1] / gw1 ok / gw2 ok

all tests are running fine without using "-n" option . I printed the attributes of config , and it does have test_system in it and i have no idea why it is failing .

Answer 1

The usual recipe of sharing data between master and the worker nodes when distributing tests with xdist is implementing the pytest_configure_node hook and later accessing the data in the following manner:

if hasattr(request.config, 'slaveinput'):
    ...  # we are in worker node
else:
    ...  # we are in master

Here is an example I tried to tie to the exception traceback you posted:

class TestSystem:
    def __init__(self):
        self.build_configuration = dict()


def pytest_configure(config):
    if hasattr(config, 'slaveinput'):  # slave
        return
    # we are on master node
    # create test system object, attach to config
    s = TestSystem()
    s.build_configuration['foo'] = 'bar'
    config.test_system = s


def pytest_configure_node(node):
    # this is the pendant of pytest_configure hook, but for worker nodes only
    # store serializable stuff from test system object in slaveinput
    node.slaveinput['test_system_serialized'] = node.config.test_system.build_configuration


def pytest_generate_tests(metafunc):
    if hasattr(metafunc.config, 'slaveinput'):  # we are in worker node
        # restore test system object using serialized data
        s = TestSystem()
        s.build_configuration = metafunc.config.slaveinput['test_system_serialized']
    else:  # master
        # simply get test system instance from config
        s = metafunc.config.test_system

    # generate tests    
    if 'test_system' in metafunc.fixturenames:
        metafunc.parametrize('test_system', [s])

Note that you can't share TestSystem instance between the master and workers, only primitive data types (strings, numbers, lists, dicts etc). This is why only the build_configuration dict is stored in slaveinput and each worker recreates its own TestSystem object from the shared data.

Example tests:

import time
import pytest


@pytest.mark.parametrize('n', range(1, 5))
def test_eggs(n, test_system):
    time.sleep(1)
    assert test_system.build_configuration['foo'] == 'bar'

Running the tests serially yields:

$ pytest -v
...
test_spam.py::test_eggs[test_system0-1] PASSED
test_spam.py::test_eggs[test_system0-2] PASSED
test_spam.py::test_eggs[test_system0-3] PASSED
test_spam.py::test_eggs[test_system0-4] PASSED

Running the test in distributed mode:

$ pytest -v -n4
...
scheduling tests via LoadScheduling

test_spam.py::test_eggs[test_system0-1] 
test_spam.py::test_eggs[test_system0-2] 
test_spam.py::test_eggs[test_system0-4] 
test_spam.py::test_eggs[test_system0-3] 
[gw1] [ 25%] PASSED test_spam.py::test_eggs[test_system0-2] 
[gw2] [ 50%] PASSED test_spam.py::test_eggs[test_system0-3] 
[gw0] [ 75%] PASSED test_spam.py::test_eggs[test_system0-1] 
[gw3] [100%] PASSED test_spam.py::test_eggs[test_system0-4]