CODEWITHSUNDEEP
python
Stephan Kristyn
Stephan Kristyn

Reputation: 15752

Python: subprocess.Popen with parameters and path

I need to execute this line in my python script:

p = subprocess.Popen(["python ~/mrjobScript.py ~/jobs/"+date+"/input/* > ~/jobs/"+date+"/output/output-"+domain+".log","--domain "+domain],stdout=subprocess.PIPE, shell=True)

However the --domain switch should come right after the mrjobScript.py. How to achieve?

Upvotes: 0

Views: 1964

Answers (2)

merlin2011
merlin2011

Reputation: 75669

Assuming that your command works on the command line, then deleting the argument stdout, changing the order of the arguments, and passing a string instead of a list should be sufficient to make it reproduce the behavior on the command line. 

import subprocess
subprocess.Popen("python ~/mrjobScript.py --domain {1} ~/jobs/{0}/input/* > ~/jobs/{0}/output/output-{1}.log ".format(date,domain), shell=True)

Note that this will start the subprocess and then go to the next line of your code. Your code will not wait for it. If you want your code to wait for it, you may want to use subprocess.call instead.

Note of Warning: It is recommended that the user of shell=True consult this answer to fully understand the implications of such usage. In particular, such applications should never let user-supplied arguments get passed directly or indirectly to the argument of a call to subprocess.Popen with shell=True without sanitization.

Upvotes: 1

Padraic Cunningham
Padraic Cunningham

Reputation: 180550

You don't use a list with shell=True, you should also redirect the output to a file to save it to a file.

import os
from subprocess import check_call
pt1 = os.path.expanduser("~/mrjobScript.py")
pt2 = os.path.expanduser("~/jobs/{}/input/*".format(date)

with open (os.path.expanduser("~/jobs/{}/output/output-{}.log".format(date,domain)),"w") as f:
     subprocess.check_call(["python",pt1,"--domain",domain,pt2],stdout=f)

Upvotes: 2

Related Questions