Bash source in a python script

Question

I am writing a wrapper python script which needs to source ./config before running some other tools. Excluding the extra functionality that my script provides, it basically does this:

$ source ./config
$ ./a-tool.sh --args
$ ./another-tool.py --arg1 --arg2

Translating to python, I can do something along the lines of this to call the two scripts:

subprocess.call(['./a-tool.sh', '--args'])
subprocess.call(['./another-tool.py', '--arg1', '--arg2'])

But I can't figure out how to source first, using the modified environment for the tools. I have read Emulating Bash 'source' in Python, but it suggests some hacks to modify the environment of running script which is not what I want.

EDIT:

To clarify, my script is currently a bash script which does a series of checks, parses some arguments, and depending on the outcome runs one or more tools as described above. The source line is not always needed, and is not always the same. However, when it is used, all the following tools need to run in the same environment (i.e. bash process). In bash, this is easy. However, I want to rewrite the script in python because some of the other functionality is just easier for me to write and maintain in python. It is purely a language choice based on preference and has nothing to do with the actual functionality. Based on the answer by @abarnert, I guess I could do something like

subprocess.call(['source', '.config', ';', './a-tool.sh', ...])

That does not look easy to debug though, so I would prefer something like:

bash = subprocess.run('bash')
bash.execute('source ./config')
bash.execute('a-tool.sh --args')

Is there any way to do this in python?

Answer 1

I agree with @Leon in the comments: switch the role of who executes what.

#!/bin/bash
source /my/conf.sh
python /my/script-1.py
python /my/script-2.py

The sourced bash config can export ENV variables and then from python you can read them with os.environ[].

This is, AFAIK, the accepted method for these sorts of problems.

Answer 2

I don't think you can do that actually. Because while using source somefile in bash, somefile will be executed in the SAME process as the main script. Thus the environment variables will be available in the main script if you put some export statements in somefile.

But, in your situation, python will create a new process (as a child of the process corresponding to current python script) and doing the source operation. After that, the process will end and be destroyed. Yes, the source operation was done successfully, but in the child process. The environment variables in parent process are available in child process, but NOT vice versa.

You should follow the link which you put in the question. In that process, environment variables are created in the current process, i.e. the process which runs your python script.

Answer 3

All source does is run the script in the context of the current bash interpreter.

The Python interpreter is not a bash interpreter, so what you're asking doesn't make any sense.^*

Usually, all source is used for is affecting environment variables, so that's what those "hacks" you linked are doing. That isn't really a hack. It assumes that the script has no effect other than to set environment variables—but that should be a good assumption; if not, then whatever else it does is just something you can't do in Python anyway.

Alternatively, you can create a bash subprocess, and use that to source the script and then run the other programs, instead of running each program directly as a subprocess of Python. (Whether by building a script on the fly, or feeding it non-tty stdin.) This is the only truly safe way to do everything that a sourced script could possibly do (because it's the only way to actually source the script).

Or, as a slightly different hack, concatenate each command onto the end of the script you want to source and run it as a separate bash subprocess. That will usually work, as long as none of the children try to change things in their parent environment or the like.

---

_{* It may have bash as a parent… but it may just as easily not… and, even if it did, that still wouldn't do what you want, unless the only thing you're trying to do is affect environment variables.}