CODEWITHSUNDEEP
pythonsubprocess
yellowcab
yellowcab

Reputation: 21

How to use Subprocess in Python

I would like to execute following shell command in python: grep 'string' file | tail -1 | cut -c 1-3

I tried:

import subprocess

i = 1
while i < 1070:
    file = "sorted." + str(i) + ".txt"
    string = "2x"
    subprocess.call(grep 'string' file | tail -1 | cut -c 1-3)
    i = i + 1

Any help would be appreciated. Thanks.

Upvotes: 0

Views: 2468

Answers (3)

hgazibara
hgazibara

Reputation: 1832

First of all, whatever you pass into the subprocess.call should be a string. Names grep, file, tail and cut are not defined in your code and you need to turn the whole expression into a string. Since the search string for the grep command should be dynamic, you need to construct the final string before passing it as argument into the function.

import subprocess

i = 1
while i < 1070:
    file = "sorted." + str(i) + ".txt"
    string = "2x"
    command_string = 'grep {0} {1} | tail -1 | cut -c 1-3'.format(string, file)
    subprocess.call(command_string)
    i = i + 1

You probably want to pass in an additional argument to subprocess.call: shell=True. The argument will make sure the command is executed through the shell.

Your command is using cut. You might want to retrieve the output of the subprocess, so a better option would be to create a new process object and use subprocess.communicate with turned out output capturing:

import subprocess

i = 1
while i < 1070:
    file = "sorted." + str(i) + ".txt"
    string = "2x"
    command_string = 'grep {0} {1} | tail -1 | cut -c 1-3'.format(string, file)

    p = subprocess.Popen(command_string, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    stdoutdata, stderrdata = p.communicate()

    # stdoutdata now contains the output of the shell commands and you can use it
    # in your program

    i = i + 1

EDIT: Here is the information on how to store the data into a text file, as requested in the comment.

import subprocess

outputs = []

i = 1
while i < 1070:
    file = "sorted." + str(i) + ".txt"
    string = "2x"
    command_string = 'grep {0} {1} | tail -1 | cut -c 1-3'.format(string, file)

    p = subprocess.Popen(command_string, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
    stdoutdata, stderrdata = p.communicate()

    # stdoutdata now contains the output of the shell commands and you can use it
    # in your program, like writing the output to a file.

    outputs.append(stdoutdata)

    i = i + 1


with open('output.txt', 'w') as f:
    f.write('\n'.join(outputs))

Upvotes: 1

mousetail
mousetail

Reputation: 8010

Subprocess expects the arguments as a string or array:

subprocess.call("grep '{}' {} | tail -1 | cut -c 1-3".format(string, file), shell=True)

shell=True is nececairy because you are using shell-specific commands like the pipe.

However, in this case it might be a lot easier to implement the entire program in pure python.

Note that if either string or file contain any special characters including spaces or quotation marks, the command will not work, and could in fact do a variety of unwanted things to your system. If you need it to work on more than these simple values, consider either a pure-python solution, setting shell=False and using the array syntax with manual piping, or some form of escaping.

Upvotes: 0

Gabio
Gabio

Reputation: 9504

Your command should be provided as a string. In addition, if you want to get the output of your command, you can use the following:

subprocess.run("grep 'string' file | tail -1 | cut -c 1-3", shell=True, capture_output=True, check=True)

where capture_output (works in Python3.7+) returns object with returncode, stdout and stderr and the check flag will raise exception if your command fails.

Upvotes: 0

Related Questions