CODEWITHSUNDEEP
macosbashpython-3.xosx-mavericks
Niklas R
Niklas R

Reputation: 16870

Strange behaviour of bash when finding executable

I'm developing a Python application that internally has to invoke the ninja command, but it couldn't find the executable with the subprocess module. When I pass shell=True, it does find the executable, but doesn't pass arguments to the process. Using os.system() worked.

subprocess.call(['ninja'] + args)              # Can't find ninja
subprocess.call(['ninja'] + args, shell=True)  # Finds & runs ninja, but the additional arguments int "args" are not passed
os.system(' '.join(shlex.quote(x) for x in ['ninja'] + args))  # Works fine

When I check where ninja is in the bash, things start to seem strange to me.

$ which ninja
$ type ninja
ninja is hashed (/Users/niklas/Bin/ninja)
$ echo $PATH
~/Bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.4/bin/:/Users/niklas/Documents/apache-maven-3.3.3/bin:/Users/niklas/Documents/grib2json-0.8.0-SNAPSHOT/bin
  1. which can't find ninja
  2. type ninja yields it is located at /Users/niklas/Bin/ninja
  3. The path /Users/niklas/Bin is not in $PATH

What's gone wrong with my setup here? Why can the bash actually find ninja, but Python only in shell=True mode, and then why are the additional arguments not passed?

Upvotes: 0

Views: 92

Answers (2)

meuh
meuh

Reputation: 12255

The problem is that ~ is a bashism, whereas python is calling os.execvp() to do the subprocess.call(), and this is obviously using execvp(), the system call, which does not do tilde expansion.

Your PATH should read /Users/niklas/Bin:.... The tilde expansion is usually done by the shell when you set PATH=~/Bin:$PATH.

So bash can find it because it re-does tilde expansion each time, but an execvp() will never find it.

And as @larsks said, shell=True needs a single string.

Upvotes: 3

larsks
larsks

Reputation: 311526

If you are setting shell=True, then your command must be a string rather than a list. Consider:

>>> import subprocess
>>> subprocess.call(['echo', 'hello'], shell=True)

0
>>> subprocess.call('echo hello', shell=True)
hello
0

If you pass a list with shell=True, only the first item of the list is considered:

>>> subprocess.call(['echo hello', 'hello'], shell=True)
hello
0
>>>

The path /Users/niklas/Bin is not in $PATH

Yes it is. It's the first item there:

$ echo $PATH
~/Bin:/usr/local/bin:...

...assuming that you are niklas.

Upvotes: 1

Related Questions