CODEWITHSUNDEEP
pythonvirtualenvinvokevirtualenvwrapper
Lucas
Lucas

Reputation: 173

Executing python script built under different virtual environment/venv?

I apologize if what I'm trying to achieve is not Pythonic - I recently moved to the language.

I have a project directory structured like so:

root
--proj1
----venv
----main.py
--proj2
----venv
----main.py

Both proj1 and proj2 run under their own virtual environments. I am trying to call proj2/main.py from proj1/main.py, whilst executing proj2/main.py under its own venv. I have tried:

import subprocess

s2_out = subprocess.check_output([sys.executable, r"..\proj2\__main__.py", "arg"])

This invokes successfully, but I am getting all manner of not found exceptions, etc. I am guessing this is the reason why.

Please let me know if there is a better approach!

Upvotes: 5

Views: 3684

Answers (3)

Hugues Sicotte
Hugues Sicotte

Reputation: 11

This is a perfectly reasonable use-case. When trying to use multiple tools in a workflow, each might have different dependencies.. and I don't feel python supports that this well. I usually stich multiple tools via bash/files, but if you want to call from within python, you have to do something like.

env = dict(os.environ)
env['PYTHONPATH'] = ... path to the library in the env...
args = [pathwithinenv/python, fullpath to python code, arg1,arg2]
p = Popen(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, 
stderr=subprocess.PIPE, env=env)

Upvotes: 1

KetZoomer
KetZoomer

Reputation: 2915

You can do this:

import subprocess
subprocess.call(["python_interpreter location (python.exe)", "python file"])

So you could do:

import subprocess
subprocess.call(["../proj2/bin/python.exe", "proj2/main.py"])

For Mac OS and Linux, the python interpreter path for a venv would be folder/bin/python.exe, or in your case ../proj2/bin/python.exe.

For Windows, the python interpreter path for a venv would be folder/scripts/python.exe.

You may need to include the full paths.

Another way to do this could be using subprocess.call, if you need the output:

import subprocess

output = subprocess.call("%s %s" %("../proj2/bin/python.exe", "proj2/main.py"))
print(output)

Both ways will work just fine :)

Upvotes: 5

Peaceful James
Peaceful James

Reputation: 2233

Hey this is not a complete answer but is how I would approach it. If you use pyenv then this would be the approach:

  1. Make a separate virtualenv for each project. pyenv virtualenv 3.8 proj1 and pyenv virtualenv 3.7 proj1 or whatever the python versions are.
  2. use pyenv local in each directory to link the dirs to the venvs
  3. cd to each directory and in each one the python env should activate. Use pip install to install the libs to each venv.
  4. Now you should have access to different python executables that use separate libs, e.g. ~/.pyenv/versions/proj1/bin/python
  5. So from proj1 code you should theoretically be able to do:
import os;
os.system("~/.pyenv/versions/proj2/bin/python ../proj2/main.py")

or something like that. I have not actually tried this but I am fairly certain it would work for using separate libs.

Here is pyenv: https://github.com/pyenv/pyenv

I will try it myself tomorrow when I am not sleepy.

Upvotes: 1

Related Questions