CODEWITHSUNDEEP
pythonpython-2.7shellcommand-linepopen
Pataquer
Pataquer

Reputation: 303

Running a complex command line in python

I would like to call a complex command line in Python and capture its output, and I don't understand how I should be doing it:

Command line that I'm trying to run is:

cat codegen_query_output.json | jq -r '.[0].code' | echoprint-inverted-query index.bin

As far as I got is:

process = subprocess.Popen(['ls', '-a'], stdout=subprocess.PIPE)
out, err = process.communicate()
print out

but this is a simple ls -a ([cmd, args]) any idea how should I run/structure my complex command line call?

Upvotes: 5

Views: 1891

Answers (2)

Jean-François Fabre
Jean-François Fabre

Reputation: 140168

The cleanest way is to create 2 subprocesses piped together. You don't need a subprocess for the cat command, just pass an opened file handle:

import subprocess

with open("codegen_query_output.json") as input_stream:
    jqp = subprocess.Popen(["jq","-r",'.[0].code'],stdin=input_stream,stdout=subprocess.PIPE)
    ep = subprocess.Popen(["echoprint-inverted-query","index.bin"],stdin=jqp.stdout,stdout=subprocess.PIPE)
    output = ep.stdout.read()
    return_code = ep.wait() or jqp.wait()

The jqp process takes the file contents as input. Its output is passed to ep input.

In the end we read output from ep to get the final result. The return_code is a combination of both return codes. If something goes wrong, it's different from 0 (more detailed return code info would be to test separately of course)

Standard error isn't considered here. It will be displayed to the console, unless stderr=subprocess.STDOUT is set (to merge with piped output)

This method doesn't require a shell or shell=True, it's then more portable and secure.

Upvotes: 3

Andomar
Andomar

Reputation: 238076

It takes a shell to interpret operators like |. You can ask Python to run a shell, and pass your command as the thing to execute:

cmd = "cat test.py | tail -n3"                                                  
process = subprocess.Popen(['bash', '-c', cmd], stdout=subprocess.PIPE)         
out, err = process.communicate()                                                
print out

Upvotes: 3

Related Questions