CODEWITHSUNDEEP
pythonbashpopen
user1115304
user1115304

Reputation: 143

Python subprocess to Bash: curly braces

I have the following Python lines:

import subprocess
subprocess.Popen("egrep -r --exclude=*{.git,.svn}* \"text\" ~/directory", stdout=subprocess.PIPE, shell=True).communicate()[0]

Unfortunately, bash completely ignores the --exclude=*{.git,.svn}* flag.

I've narrowed down the problem to the curly braces. --exclude=*.git* will work through python's popen, but the moment curly braces are introduced, I'm left helpless. Any suggestions?

Note: I tried running the command using Python's command library, it produces the exact same output -- and the exact same ignored --exclude flag.

Upvotes: 6

Views: 2897

Answers (4)

Denis Arnaud
Denis Arnaud

Reputation: 388

From a Python Popen perspective, what you have wrotten works, provided you capture the output in a Python variable:

import subprocess
myOutput = subprocess.Popen("egrep -r --exclude=*{.git,.svn}* \"text\" ~/directory", stdout=subprocess.PIPE, shell=True).communicate()[0]
print "Output: ", myOutput

I have tested in a terminal with the Bash as default Shell command, and it works well.

Note that 'grep -E' should be preferred to 'egrep', which is now deprecated.

And you certainly know that \ is also the escape character for the Bash, don't you? I mean, '*' and the curly braces are consumed by the Bash, and are therefore not handed over to grep. You should therefore escape them.

grep -Er --exclude=\*\{.git,.svn\}\* \"text\" ~/directory

Upvotes: 0

David K. Hess
David K. Hess

Reputation: 17246

You need to quote that expression to keep bash from evaluating it against your current working directory when you fire it off. You also have a bug with your search term assuming you are looking for "text" (with the quotations). Your escaping gets the quotes into the Python string but needs to be done again to have the shell see them.

I.e. ... --exclude='*{.git,.svn}*' \\\"text\\\" ...

Upvotes: 0

Douglas Leeder
Douglas Leeder

Reputation: 53310

I would guess that it might be shell escaping?

It might be best to do your own splitting up of the arguments, and avoid the shell entirely?

import subprocess
subprocess.Popen(["egrep","-r","--exclude=*{.git,.svn}*","text","~/directory"], stdout=subprocess.PIPE).communicate()[0]

NB: You might have to expand ~, I'm not sure.

Or if bash is supposed to be expanding the braces then you could do it in python:

excludes = ['.git','.svn']
command = ['egrep','-r']
for e in excludes:
    command.append('--exclude=*%s*'%e)
command += ["text","~/directory"]
subprocess.Popen(command, stdout=subprocess.PIPE).communicate()[0]

Upvotes: 2

Rob Wouters
Rob Wouters

Reputation: 16327

When you pass shell=True, python translates the command to /bin/sh -c <command> (as described here). /bin/sh apparently does not support the curly braces expansion. You can try the following instead:

import subprocess
subprocess.Popen(["/bin/bash", "-c", "egrep -r --exclude=*{.git,.svn}* \"text\" ~/directory"], stdout=subprocess.PIPE).communicate()[0]

Upvotes: 3

Related Questions