CODEWITHSUNDEEP
pythonsubprocessstdoutstdinio-redirection
Alex Reynolds
Alex Reynolds

Reputation: 96927

How to execute process in Python where data is written to stdin?

I have a flag in my Python script which specifies whether I setup and use an external process or not. This process is a command called my_command and it takes data from standard input. If I was to run this on the command-line, it would be something like:

$ my_command < data > result

I want to use a Python script to generate lines of data by modifying standard input and feeding it to my_command.

I'm doing something like this:

import getopt, sys, os, stat, subprocess

# for argument's sake, let's say this is set to True for now
# in real life, I use getopt.getopt() to decide whether this is True or False
useProcess = True

if useProcess:
    process = subprocess.Popen(['my_command'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)

for line in sys.stdin:
    # parse line from standard input and modify it
    # we store the result in a variable called modified_line
    modified_line = line + "foo"

    # if we want to feed modified_line to my_command, do the following:
    if useProcess:
        process.stdin.write(modified_line)

    # otherwise, we just print the modified line
    else:
        print modified_line

However, my_command behaves as if it does not receive any data and quits with an error state. What am I doing wrong?

EDIT

Let's say my Python script is called my_Python_script. Let's say I would normally pass my_command a file called data over standard input:

$ my_command < data > result

But now I'm passing it to my_Python_script instead:

$ my_Python_script < data > some_other_result

I want my_Python_script to conditionally set up a subprocess that runs my_command on the contents of data (which are modified by my_Python_script before being passed to my_command). Does this make more sense?

If I was using bash as a scripting language, I would conditionally decide to run one of two functions. One would pipe lines of data to my_command. The other would not. Can this be done with Python?

Upvotes: 9

Views: 5731

Answers (3)

Hai Vu
Hai Vu

Reputation: 40688

After writing to the stdin, you need to close it:

    process.stdin.write(modified_line)
    process.stdin.close()

Update

I failed to notice that the process.stdin.write() was executed in a for loop. In which case, you should move the process.stdin.close() to outside the loop.

Also, Raymond mentioned that we should call process.wait() as well. So the updated code should be:

for ...
    process.stdin.write(modified_line)

process.stdin.close()
process.wait()

Upvotes: 10

Raymond Tau
Raymond Tau

Reputation: 3469

In addition to process.stdin.close() as mentioned by @HaiVu, did you do process.wait() to wait for the command to finish before getting the result?

Upvotes: 3

Jeremy Blalock
Jeremy Blalock

Reputation: 2619

It seems like you may be confusing arguments and stdin. Your command should be 

$ <data> | mycommand result

with data being passed in once the command is called.

Taking an input is done with the raw_input builtin function. (http://docs.python.org/2/library/functions.html)

Upvotes: 0

Related Questions