CODEWITHSUNDEEP
pythonbashenvironment-variables
nish
nish

Reputation: 1058

Set bash environment variable from python script

I am running 4 Python scripts which in short should set environment variables.

Point is, using os.environ['TEST'] (say) sets it only for the scope of the Python script that is running it. 

$ python test.py arg1
$ echo $TEST    < ------------

This returns blank. Whereas printing from within the Python script shows a value in TEST

os.environ.get('TEST')

How does one set environment variables from python script?

As a workaround to put it another way, (alternate question) how does one share data between Python scripts?

Upvotes: 0

Views: 2201

Answers (3)

cdarke
cdarke

Reputation: 44344

The environment block is an area of memory unique to each process. The contents are determined by the way the process is started. By default the parent process's environment block is copied to a child process. The child is free to alter its own environment block but cannot alter it's parent's environment.

The bottom line is that environment values are only passed in one direction, from parent to child.

This applies to all processes, regardless of which language the programs they use are written in.

"using os.environ['TEST'] (say) sets it only for the scope of the Python script that is running it." - correct

"How does one set environment variables from python script?" - in theory a debugger, which would probably have to be written in C, could be attached to a process and alter the block of memory. That is difficult, dangerous (you could overwrite the wrong memory), a huge security risk, and a maintenance nightmare. You really don't want to go there.

Sharing data between processes is a wide question, it doesn't really matter which language you are using - the principles are the same.

It depends on the type of data (binary or text), the volume of data (a few bytes or megabytes), if the data is passed in both directions (synchronisation might be needed), if the data has to be persistent (a file or database might be required), and probably a million other things I have not thought of.

You need to be more specific about which data is shared under what circumstances, including who does the writing and who does the reading.

Upvotes: 1

Amadan
Amadan

Reputation: 198304

It is not possible to set an environment variable for a parent (or ancestor) process environment.

Some bash scripts go around this requirement by executing with source (which runs them inside the current bash, instead of spawning a bash process), but this is obviously never going to work with a python script, as you can't execute a python script inside the bash process itself without spawning a python subprocess.

Some other programs (such as ssh-agent) output the necessary assignments to the standard output in such a way that their output can be run through eval in bash; this works, though it requires the caller's cooperation (i.e. running eval $(ssh-agent) instead of just ssh-agent).

About your sub-question - There are many ways to share data between processes: command-line arguments, pipes, named pipes, files, sockets, signals, databases (both SQL and no-SQL), other services (web and otherwise)... You would need to specify your scenario in more detail if you ask us to pick the most appropriate one. But that should really be a new question, if it comes to that.

Upvotes: 4

wim
wim

Reputation: 362478

You seem to have a complete misunderstanding about how processes and environments work. 

$ python test.py arg1
$ echo $TEST

The first line will create a new process, and it's an entirely separate process from the shell itself. Though the values of environment variables may be inherited from the environment of the parent process, the new process has its own environment.

By the time the second line executes, the process and environment from the first line no longer even exist! You can not easily modify the environment of the shell's process from within the Python process.

As a workaround to put it another way, (alternate question) how does one share data between python scripts?

Not via the environment - after the Python process completes, i.e. when your script exits, its environment has been torn down. The "alternate question" is perhaps too broad, since there are many ways to share data between Python processes, but I suggest you start by reading/writing the data to/from a file.

Upvotes: 4

Related Questions