CODEWITHSUNDEEP
python
Ankur Agarwal
Ankur Agarwal

Reputation: 24758

python subprocess hang on reading a pipe

I have this but the subprocess reading from pipe at the end hangs:

$cat waitforinput.sh
#!/bin/sh


while read line
do
echo $line
done                    


>>> p1 = subprocess.Popen(["/home/abc/waitforinput.sh", "-v"], shell=True, executable=None,stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> 
>>> 
>>> p1.stdin.write('This is a good idea\n')
>>> p1.stdin.write('This is a good idea\n')
>>> p1.stdin.write('This is a good idea\n')
>>> p1.stdin.write('This is a good idea\n')
>>> p1.stdin.write('This is a good idea\n')
>>> 
>>> 
>>> p1.stdin.flush()
>>> 
>>> for i in p1.stdout:
...     print i 
...

What should I do so that it does not hang?

Upvotes: 2

Views: 2903

Answers (2)

falsetru
falsetru

Reputation: 368954

Instead of flush(), call p1.stdin.close().

...
p1.stdin.write('This is good idea\n')
p1.stdin.write('This is good idea\n')

p1.stdin.close()

for i in p1.stdout:
    print i

UPDATE

Replace stdout-iteration with while-loop with stdout.readline()

Refer python manpage -u part:

Force stdin, stdout and stderr to be totally unbuffered. On systems where it matters, also put stdin, stdout and stderr in binary mode. Note that there is internal buffering in xread‐ lines(), readlines() and file-object iterators ("for line in sys.stdin") which is not influenced by this option. To work around this, you will want to use "sys.stdin.readline()" inside a "while 1:" loop.

p1.stdin.flush()

while True:
    line = p1.stdout.readline()
    if not line:
        break
    print line

You will get output, but without close(), the script will not end. Anyway you should use close().

Upvotes: 3

tdelaney
tdelaney

Reputation: 77337

The problem is that waitforinput.sh is buffering its output. falsetru's solution works because the close causes the script to exit and that flushes its output buffers. This is the normal way to handle pipelined commands.

If you want the output interactively, you can use the pty module or pexpect to trick the script into thinking that it is writing to a terminal. Then, its output will only be line buffered.

Upvotes: 1

Related Questions