CODEWITHSUNDEEP
pythonpytest
Jakob
Jakob

Reputation: 20811

Setup remote environment when using xdist

I'm currently working on a project involving tests to be run at a remote host (bash). Unfortunately, the remote python interpreter does not respect the available site-packages (it's an embedded one: abaqus python (2.6)). Using the PYTHONPATH variable, however, works to specify local installations and makes additional packages available. Hence, on the remote machine I simply add a respective line to my .bashrc file.

Unfortunately, when distributing tests using xdist only a "bare" bash is invoked, without any profile specific rcs loaded. Thus, the tests fail with some import errors as argparse, which is required by pytest is not available.

Is there a way the setup a remote host before it starts executing any pytest code (which requires argparse)? In other words is there a way to add environment variables on the hosts before the pytest imports start?

I tried using fixtures with session scope and autouse=True which (of course) didn't work. Moreover I tried something like

# in conftest.py
import sys    
def pytest_configure_node():
    sys.path.insert(1, "/somepath/")
    print sys.path

This looks like it is executed on the remote host but sys.path remains the hosts one and the argparse module still cannot be imported.

I start the tests using 

py.test --tx ssh=user@server//python="abaqus613 python" -vs --dist=each --rsyncdir foo

This starts the right python interpreter (Python 2.6.2 for Abaqus 6.13-2), but fails with

ImportError: No module named argparse

Upvotes: 9

Views: 2844

Answers (2)

Jakob
Jakob

Reputation: 20811

I finally figured out a quite hackish but reasonably working way. It is possible to specify a series of commands in the python call, hence I source a script which setups the environment before calling abaqus python.

The setup script (setup.sh), located on the remote machine, looks like: 

export PYTHONPATH=/path/to/libraries

and the complete call is now

py.test --tx ssh=user@server//python="source setup.sh;abaqus613 python" -vs --dist=each --rsyncdir foo

This way I get the necessary imports working in abaqus python.

Upvotes: 1

Anatoly Bubenkov
Anatoly Bubenkov

Reputation: 696

My plugin https://github.com/pytest-dev/pytest-cloud should help you

it automatically rsyncs python virtual environment to the slaves, one requirement is that your virtualenv folder should be inside of the folder where test folder is

after rsync it also activates the virtual environment so all your dependencies are there properly

Upvotes: 0

Related Questions