CODEWITHSUNDEEP
pythonsubprocess
Codelearner
Codelearner

Reputation: 251

Using piping with subprocess.Popen

I want pipe the output of one command to another. Command 2 has to be run after the command 1 execution is completed and so on. I tried using 'wait' of 'subprocess.Popen()'. But somehow it is not working.

Can someone help me on this?

Code used:

proc= subprocess.Popen(['python27','scriptA.py',file],cwd="C:\pythonPrograms\\",stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
proc.wait()
sortedop= subprocess.Popen(['sort'],cwd="C:\pythonPrograms\\",stdout=subprocess.PIPE,stdin=proc.stdout, stderr=subprocess.STDOUT)
sortedop.wait()
countReducer= subprocess.Popen(['python27', 'scriptB.py'],cwd="C:\pythonPrograms\\",stdout=subprocess.PIPE,stdin=sortedop.stdout, stderr=subprocess.STDOUT)
countReducer.wait()
fd = open(file+".json", 'w')
countpostprocesser= subprocess.Popen(['python27', 'scriptC.py'],cwd="C:\pythonPrograms\\",stdout=fd,stdin=countReducer.stdout,stderr=subprocess.STDOUT,shell=True)
countpostprocesser.wait()
fd.close()

Upvotes: 0

Views: 245

Answers (1)

tdelaney
tdelaney

Reputation: 77347

If you really want one script to complete before the next one starts and the output data isn't too large, you can read the data into memory:

proc= subprocess.Popen(['python27','scriptA.py',file],cwd="C:\\pythonPrograms\\",stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = proc.communicate()
sortedop= subprocess.Popen(['sort'],cwd="C:\pythonPrograms\\",stdout=subprocess.PIPE,stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
out, err = sortedop.communicate(out)
countReducer= subprocess.Popen(['python27', 'scriptB.py'],cwd="C:\pythonPrograms\\",stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
out, err = countReducer.communicate(out)
fd = open(file+".json", 'w')
countpostprocesser= subprocess.Popen(['python27', 'scriptC.py'],cwd="C:\pythonPrograms\\",stdout=fd, stderr=subprocess.STDOUT)
out, err = countpostprocesser.communicate(out)

Upvotes: 1

Related Questions