CODEWITHSUNDEEP
pythonbashsubprocess
jakun
jakun

Reputation: 674

The difference between Python's subprocess.Popen with shell=True and a real shell

When I open a command line I want it to start in the same working directory where the currently active window is. Therefore I have written a python script which figures out the correct path. It takes as argument a command which opens a terminal. In this argument it replaces a placeholder with the desired directory and executes the command with

subprocess.Popen(cmd, stderr=subprocess.PIPE, shell=True)

The string cmd is correctly assembled to:

xterm -e 'cd '\''/mnt/data/software/computer/tools/i3'\''; /usr/bin/bash'

If I execute this command in bash it opens xterm in the correct directory as desired. If I execute this command from the python script xterm does not open.

Where is the difference?

Additional information:

I am using Python 3.6.5.

echo $SHELL returns /bin/bash.

The keybinding to the script in my i3 config:

bindsym $mod+Return exec "/mnt/data/software/computer/tools/i3/i3_launch_cwd.sh \\"xterm -e 'cd %{cwd}; /usr/bin/bash'\\""

(This shell script is just a simple wrapper which redirects stderr to a log file for debugging)

After executing the following command in bash xterm does open even from python:

xrdb -merge -I$HOME ~/.Xresources

The relevant line in Xresources is

xterm*faceName: DejaVu Sans Mono Book

Why does that make a difference?

Solution: Thanks to Charles Duffy's comments I have found the problem. In my python script I am redirecting stderr with stderr=subprocess.PIPE but I forgot to read stderr. Before loading my Xresources file which sets the font xterm prints a warning to stderr that a font can not be loaded. xterm has a very small buffer. Because my program is not reading stderr the buffer is not cleared and xterm is blocked.

Replacing 

subprocess.Popen(cmd, stderr=subprocess.PIPE, shell=True)

with 

p = subprocess.Popen(cmd, stderr=subprocess.PIPE, shell=True)
out, err = p.communicate()
sys.stderr.write(err)

Would solve the problem. But because I am not doing anything with stderr there is no need to redirect it in the first place. So instead I am now using

subprocess.Popen(cmd, shell=True)

Upvotes: 1

Views: 1370

Answers (1)

Charles Duffy
Charles Duffy

Reputation: 295363

Don't use shell=True; it's not appropriate to your use case.

import subprocess
try:
    from pipes import quote # Python 2
except ImportError:
    from shlex import quote # Python 3

dir='/mnt/data/software/computer/tools/i3'
p = subprocess.Popen(['xterm', '-e', 'cd %s && exec bash' % quote(dir)])

However, you can simplify this by letting subprocess.Popen set the directory for you:

dir='/mnt/data/software/computer/tools/i3'
p = subprocess.Popen(['xterm', '-e', 'bash'], cwd=dir)

That said, shell=True is 100% equivalent to running sh -c '...command...', where your command's literal text is in ...command...; this is exactly what it does in practice.

Note that I removed stderr=subprocess.PIPE above. You can add that back in, but only if you have your code actually read content written to stderr, as by calling communicate().

Upvotes: 4

Related Questions