How can I resolve an error running pytest in parallel via xdist in bitbucket pipelines

Question

We execute our unit tests for our python repo within a bitbucket pipeline. Recently we started using the pytest-xdist (https://pypi.org/project/pytest-xdist/) package and successfully run our unit tests in parallel when running locally via the coverage run -m pytest -n auto command. This executes all unit tests across all available cpus on our local machines. However when we run the tests with the same command via a bitbucket pipeline we encounter the following exception. Is there any advice on how to resolve this?

+ coverage run -m pytest -n auto
============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.4.3, py-1.10.0, pluggy-0.13.1
rootdir: /opt/atlassian/pipelines/agent/build
plugins: celery-4.4.7, forked-1.3.0, xdist-2.2.1, mock-3.5.1
gw0 I / gw1 I / gw2 I / gw3 I
INTERNALERROR> def worker_internal_error(self, node, formatted_error):
INTERNALERROR>         """
INTERNALERROR>         pytest_internalerror() was called on the worker.
INTERNALERROR>     
INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr, which can't
INTERNALERROR>         be serialized, so we go with a poor man's solution of raising an exception
INTERNALERROR>         here ourselves using the formatted message.
INTERNALERROR>         """
INTERNALERROR>         self._active_nodes.remove(node)
INTERNALERROR>         try:
INTERNALERROR> >           assert False, formatted_error
INTERNALERROR> E           AssertionError: Traceback (most recent call last):
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR> E                 session.exitstatus = doit(config, session) or 0
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 246, in _main
INTERNALERROR> E                 config.hook.pytest_collection(session=session)
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR> E                 return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR> E                 firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 257, in pytest_collection
INTERNALERROR> E                 return session.perform_collect()
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 453, in perform_collect
INTERNALERROR> E                 self.config.pluginmanager.check_pending()
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 277, in check_pending
INTERNALERROR> E                 % (name, hookimpl.plugin),
INTERNALERROR> E             pluggy.manager.PluginValidationError: unknown hook 'pytest_warning_recorded' in plugin <__channelexec__.WorkerInteractor object at 0x7f11772d84d0>
INTERNALERROR> E           assert False
INTERNALERROR> 
INTERNALERROR> /usr/local/lib/python3.7/site-packages/xdist/dsession.py:187: AssertionError
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/_pytest/main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/xdist/dsession.py", line 112, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/xdist/dsession.py", line 135, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/xdist/dsession.py", line 175, in worker_workerfinished
INTERNALERROR>     self._active_nodes.remove(node)
INTERNALERROR> KeyError: <WorkerController gw0>

Thanks in advance for any help!

Answer 1

I got the same issue but I can not solve this issue after upgrading pytest to 6.2.2

Answer 2

Found the issue! I was running with an old version of pytest. Pinning the latest version of pytest (which at time of writing is pytest==6.2.2 to my requirements.txt solved the issue. A subsequent issue was also that the more-itertools package was not built into the python enviornment of the bitbucket pipeline container I had configured so pinned this also to requirements