CODEWITHSUNDEEP
pythonsubprocesspopen
Oren_H
Oren_H

Reputation: 4811

Python subprocess/Popen with a modified environment

I believe that running an external command with a slightly modified environment is a very common case. That's how I tend to do it:

import subprocess, os
my_env = os.environ
my_env["PATH"] = "/usr/sbin:/sbin:" + my_env["PATH"]
subprocess.Popen(my_command, env=my_env)

I've got a gut feeling that there's a better way; does it look alright?

Upvotes: 451

Views: 396751

Answers (10)

Flimm
Flimm

Reputation: 151231

In newer versions of Python, you can use the env keyword argument of the run function, like this:

import subprocess
import os

subprocess.run(["mycommand"], env=os.environ | {"FOO": "bar"})

Because I wanted the process to inherit the existing environment variables, I merged os.environ with my own dictionary of overridden environment variables using the union operator, which works in Python 3.9+.

Upvotes: 7

Daniel Burke
Daniel Burke

Reputation: 6324

I think os.environ.copy() is better if you don't intend to modify the os.environ for the current process:

import subprocess, os
my_env = os.environ.copy()
my_env["PATH"] = f"/usr/sbin:/sbin:{my_env['PATH']}"
subprocess.Popen(my_command, env=my_env)

Upvotes: 630

Bertrand Michel
Bertrand Michel

Reputation: 17

This could be a solution :

new_env = dict([(k,(':'.join([env[k], v]) if k in env else v)) for k,v in os.environ.items()])

Upvotes: -2

MFB
MFB

Reputation: 19837

To temporarily set an environment variable without having to copy the os.envrion object etc, I do this:

process = subprocess.Popen(['env', 'RSYNC_PASSWORD=foobar', 'rsync', \
'rsync://[email protected]::'], stdout=subprocess.PIPE)

Upvotes: 21

skovorodkin
skovorodkin

Reputation: 10284

With Python 3.5 you could do it this way:

import os
import subprocess

my_env = {**os.environ, 'PATH': '/usr/sbin:/sbin:' + os.environ['PATH']}

subprocess.Popen(my_command, env=my_env)

Here we end up with a copy of os.environ and overridden PATH value.

It was made possible by PEP 448 (Additional Unpacking Generalizations).

Another example. If you have a default environment (i.e. os.environ), and a dict you want to override defaults with, you can express it like this:

my_env = {**os.environ, **dict_with_env_variables}

Upvotes: 51

skyking
skyking

Reputation: 14400

That depends on what the issue is. If it's to clone and modify the environment one solution could be:

subprocess.Popen(my_command, env=dict(os.environ, PATH="path"))

But that somewhat depends on that the replaced variables are valid python identifiers, which they most often are (how often do you run into environment variable names that are not alphanumeric+underscore or variables that starts with a number?).

Otherwise you'll could write something like:

subprocess.Popen(my_command, env=dict(os.environ, 
                                      **{"Not valid python name":"value"}))

In the very odd case (how often do you use control codes or non-ascii characters in environment variable names?) that the keys of the environment are bytes you can't (on python3) even use that construct.

As you can see the techniques (especially the first) used here benefits on the keys of the environment normally is valid python identifiers, and also known in advance (at coding time), the second approach has issues. In cases where that isn't the case you should probably look for another approach.

Upvotes: 125

derigible
derigible

Reputation: 964

I know this has been answered for some time, but there are some points that some may want to know about using PYTHONPATH instead of PATH in their environment variable. I have outlined an explanation of running python scripts with cronjobs that deals with the modified environment in a different way (found here). Thought it would be of some good for those who, like me, needed just a bit more than this answer provided.

Upvotes: 2

Noufal Ibrahim
Noufal Ibrahim

Reputation: 72815

The env parameter accepts a dictionary. You can simply take os.environ, add a key (your desired variable) (to a copy of the dict if you must) to that and use it as a parameter to Popen.

Upvotes: 6

SilentGhost
SilentGhost

Reputation: 319899

you might use my_env.get("PATH", '') instead of my_env["PATH"] in case PATH somehow not defined in the original environment, but other than that it looks fine.

Upvotes: 26

Andrew Aylett
Andrew Aylett

Reputation: 40750

In certain circumstances you may want to only pass down the environment variables your subprocess needs, but I think you've got the right idea in general (that's how I do it too).

Upvotes: 0

Related Questions