CODEWITHSUNDEEP
pythonpython-3.xpipesubprocess
waka-waka-waka
waka-waka-waka

Reputation: 1045

python subprocess multiple stdin.write and stdout.read

Thanks for taking time to answer the question. I am playing around with Python 3.4 and I have two simple python programs. One, a program called test.py that takes a user input and prints something.

while True:
    print("enter something...")
    x = input()
    print(x)
    time.sleep(1)

To send input to this program, I have another program that uses subprocess:

from subprocess import Popen, PIPE

cat = Popen('python test.py', shell=True, stdin=PIPE, stdout=PIPE)
cat.stdin.write("hello, world!\n")
cat.stdin.flush()
print(cat.stdout.readline())

cat.stdin.write("and another line\n")
cat.stdin.flush()
print(cat.stdout.readline())

However when I run the above program, I get an error:

enter something...

hello, world!
Traceback (most recent call last):
  File "/opt/test.py", line 9, in <module>
    x = input()
EOFError: EOF when reading a line
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

And if I replace test.py with a standard linux command like 'cat', things work as expected.

Is there any way I send multiple stdin writes and read multiple stdout back?

Upvotes: 5

Views: 8893

Answers (1)

jfs
jfs

Reputation: 414865

In general, you should use pexpect for interactive programs (dialog-based interactions).

Your specific issue might be caused by a python version mismatch (you think your code is executed using Python 3 while actually it might be executed using Python 2). The second issue (EOFError) is expected: either catch it in the child script or provide a signal for the child to exit (I use an empty line for that in the code example below).

Here's a Python 3 code that fails loudly on Python 2:

#!/usr/bin/env python3
import sys
from subprocess import Popen, PIPE

with Popen([sys.executable, '-u', 'test.py'], stdin=PIPE, stdout=PIPE,
           universal_newlines=True, bufsize=1) as cat:
    for input_string in ["hello, world!", "and another line", ""]:
        print(input_string, file=cat.stdin, flush=True)
        print(cat.stdout.readline(), end='')

Note:

And here's the corresponding test.py:

#!/usr/bin/env python3
import time

while True:
    x = input("enter something...")
    if not x: # exit if the input is empty
        break
    print(x)
    time.sleep(1)

Output

enter something...hello, world!
enter something...and another line
enter something...

Note: there is no new line after "enter something..."

It works but it is fragile, read Q: Why not just use a pipe (popen())? and use pexpect instead.

If the input is finite and it doesn't depend on the output then you could pass it all at once:

#!/usr/bin/env python3
import sys
from subprocess import check_output

output = check_output([sys.executable, 'test.py'],
                      input="\n".join(["hello, world!", "and another line"]),
                      universal_newlines=True)
print(output, end='')

This version requires that the child handles EOF properly:

#!/usr/bin/env python3
import time

while True:
    try:
        x = input("enter something...")
    except EOFError:
        break # no more input

    print(x)
    time.sleep(1)

The output is the same (as shown above).

Upvotes: 5

Related Questions