CODEWITHSUNDEEP
pythonpython-2.7subprocess
sherpaurgen
sherpaurgen

Reputation: 3274

How to correctly escape special characters in python subprocess?

Im trying to run this bash command using python subprocess 

find /Users/johndoe/sandbox -iname "*.py" | awk -F'/' '{ print $NF}'

output:-

helld.xl.py
parse_maillog.py
replace_pattern.py
split_text_match.py
ssh_bad_login.py

Here is what i have done in python2.7 way, but it gives the output where awk command filter is not working

>>> p1=subprocess.Popen(["find","/Users/johndoe/sandbox","-iname","*.py"],stdout=subprocess.PIPE)

>>> p2=subprocess.Popen(['awk','-F"/"','" {print $NF} "'],stdin=p1.stdout,stdout=subprocess.PIPE)

>>>p2.communicate()
('/Users/johndoe/sandbox/argparse.py\n/Users/johndoe/sandbox/custom_logic_substitute.py\n/Users/johndoe/sandbox/finditer_html_parse.py\n/Users/johndoe/sandbox/finditer_simple.py\n/Users/johndoe/sandbox/group_regex.py\n/Users/johndoe/sandbox/helo.py\n/Users/johndoe/sandbox/newdir/helld.xl.py\n/Users/johndoe/sandbox/parse_maillog.py\n/Users/johndoe/sandbox/replace_pattern.py\n/Users/johndoe/sandbox/split_text_match.py\n/Users/johndoe/sandbox/ssh_bad_login.py\n', None)

I could also get output by using p1 alone here like below,but i cant get the awk working here

list1=[]
result=p1.communicate()[0].split("\n")
for item in res:
    a=item.rstrip('/').split('/')
    list1.append(a[-1])
print list1

Upvotes: 2

Views: 17639

Answers (3)

tripleee
tripleee

Reputation: 189507

You are incorrectly passing in shell quoting (and extra shell quoting which isn't even required by the shell!) when you're not invoking a shell. Don't do that.

p2=subprocess.Popen(['awk', '-F/', '{print $NF}'], stdin=...

When you have shell=True you need extra quotes around some arguments to protect them from the shell, but there is no shell here, so putting them in is incorrect, and will cause parse errors by Awk.

However, you should almost never need to call Awk from Python, especially for trivial tasks which Python can easily do natively:

list1 = [line.split('/')[-1]
            for line in subprocess.check_output(
                ["find", "/Users/johndoe/sandbox",
                 "-iname", "*.py"]).splitlines()]

In this particular case, note also that GNU find already has a facility to produce this result directly:

list1 = subprocess.check_output(
    ["find", "/Users/johndoe/sandbox",
     "-iname", "*.py", "-printf", "%f\\n"]).splitlines()

Upvotes: 3

pankaj mishra
pankaj mishra

Reputation: 2615

if you don't have any reservation using shell=True , then this should be pretty simple solution

from subprocess import Popen
import subprocess
command='''
find /Users/johndoe/sandbox -iname "*.py" | awk -F'/' '{ print $NF}'
'''
process=Popen(command,shell=True,stdout=subprocess.PIPE)
result=process.communicate()
print result

Upvotes: 0

mcchucklezz
mcchucklezz

Reputation: 376

Use this: p2.communicate()[0].split("\n").

It will output a list of lines.

Upvotes: 0

Related Questions