CODEWITHSUNDEEP
linuxcondaqsub
BGreen
BGreen

Reputation: 450

Using a conda environment to run a job in qsub

I'd like to use qsub to submit a job which runs some Python code which uses Numpy. Numpy is installed in a conda virtual environment which I activate upon logging into the cluster, and I can import it if I simply call import numpy in the Python command line interpreter.

However, when I submit a job with qsub, it tries to run the job without using that environment. After some testing it seems that I can get qsub to run things in the right environment if I pass in all variables with the -V option.

However, this muddles up other parts of the scripts. It would be much nicer if I could pass in only what I need to get qsub to run in the right environment. How can I do this?

P.S. The solution presented here did not work for me; the error was Unable to locate a modulefile for 'numpy'. I'd assume it's not in the current path but I'm not sure where it.

Upvotes: 1

Views: 2315

Answers (1)

BGreen
BGreen

Reputation: 450

I've got a solution now. In the script that is submitted with qsub I've added two lines to have it reload the environment.

source [Intel Parallel Studios script that sets up conda environment variables]
source activate my_root

Upvotes: 1

Related Questions