CODEWITHSUNDEEP
pythonsubprocess
ihf
ihf

Reputation: 3

subprocess.call with command having embedded spaces and quotes

I would like to retrieve output from a shell command that contains spaces and quotes. It looks like this:

import subprocess
cmd = "docker logs nc1 2>&1 |grep mortality| awk '{print $1}'|sort|uniq"
subprocess.check_output(cmd)

This fails with "No such file or directory". What is the best/easiest way to pass commands such as these to subprocess?

Upvotes: 0

Views: 212

Answers (1)

tripleee
tripleee

Reputation: 189317

The absolutely best solution here is to refactor the code to replace the entire tail of the pipeline with native Python code.

import subprocess
from collections import Counter

s = subprocess.run(
    ["docker", "logs", "nc1"],
    text=True, capture_output=True, check=True)
count = Counter()
for line in s.stdout.splitlines():
    if "mortality" in line:
        count[line.split()[0]] +=  1
for count, word in count.most_common():
    print(count, word)

There are minor differences in how Counter objects resolve ties (if two words have the same count, the one which was seen first is returned first, rather than by sort order), but I'm guessing that's unimportant here.

I am also ignoring standard output from the subprocess; if you genuinely want to include output from error messages, too, just include s.stderr in the loop driver too.

However, my hunch is that you don't realize your code was doing that, which drives home the point nicely: Mixing shell script and Python raises the mainainability burden, because now you have to understand both shell script and Python to understand the code.

(And in terms of shell script style, I would definitely get rid of the useless grep by refactoring it into the Awk script, and probably also fold in the sort | uniq which has a trivial and more efficient replacement in Awk. But here, we are replacing all of that with Python code anyway.)

If you really wanted to stick to a pipeline, then you need to add shell=True to use shell features like redirection, pipes, and quoting. Without shell=True, Python looks for a command whose file name is the entire string you were passing in, which of course doesn't exist.

Upvotes: 1

Related Questions