CODEWITHSUNDEEP
pythonunixsubprocess
user248237
user248237

Reputation:

invoking pipe command with Popen in Python

What is the correct way to invoke a command containing multiple pipes to Popen so that its output can be read? I tried:

Popen(shlex.split("mycmd arg1 | mysecondcmd - | thirdcmd -", stdout=PIPE)")

But I don't believe shlex.split is right here. What's the correct syntax?

Upvotes: 1

Views: 2801

Answers (2)

unutbu
unutbu

Reputation: 879123

Using the sh module, pipes become function composition:

import sh
output = sh.thirdcmd(sh.mysecondcmd(sh.mycmd("arg1")))

If you want to do it with subprocess without shell = True, there is an example in the docs which shows how to write shell pipelines using subprocess.Popen. Note that you are supposed to close the proc.stdouts so that SIGPIPEs can be received properly:

import subprocess
proc1 = subprocess.Popen(shlex.split('mycmd arg1'), stdout = subprocess.PIPE)
proc2 = subprocess.Popen(shlex.split('mysecondcmd'), stdin = proc1.PIPE,
                         stdout = subprocess.PIPE)
proc3 = subprocess.Popen(shlex.split('thirdcmd'), stdin = proc2.PIPE,
                         stdout = subprocess.PIPE)

# Allow proc1 to receive a SIGPIPE if proc2 exits.
proc1.stdout.close()
# Allow proc2 to receive a SIGPIPE if proc3 exits.
proc2.stdout.close()
out, err = proc3.communicate()

This might look like a lot more work than using shell = True. The reason why you might want to avoid shell = True is because it can be a security risk (page down to the "Warning" box), especially if you are running a command supplied by a (potentially malicious) user.

Upvotes: 3

mgilson
mgilson

Reputation: 309831

You have a few options -- You can pass shell=True:

Popen('command1 | command2 | command3',shell=True)

Or, you can break it up into a bunch of Popen calls hooking their stdout to the next Popen's stdin as demonstrated in the documentation.

Upvotes: 6

Related Questions