CODEWITHSUNDEEP
pythonsubprocessstdinpopen
user1379351
user1379351

Reputation: 785

Subprocess popen: Why do all writes happen at once in child process?

I have two scripts, one controlling the other and communicating to it via stdin. The parent script:

import subprocess
import time

p = subprocess.Popen(['python','read_from_stdin.py'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)

for i in range(0,10):
    p.stdin.write(str(i))
    p.stdin.write('\r\n') # \n is not sufficient on Windows
    p.stdin.flush()
    print i
    time.sleep(1)

p.stdin.close()

The child script (called 'read_from_stdin.py'):

import sys
import datetime

with open(datetime.datetime.now().strftime('%Y-%m-%d-%H-%M-%S') + '.txt','w') as f:

    for line in sys.stdin:
        f.write(datetime.datetime.now().isoformat() + ' ' + line)

In the file that's created by the child script all the inputs have the same timestamp, despite being written a second apart by the parent script, and despite the use for flush().

Upvotes: 2

Views: 93

Answers (2)

user1379351
user1379351

Reputation: 785

EDIT: As per Karoly Horvath's comment below, it's not the case that it waits for for EOF, but there's buffering. The different child script below does work as expected.

I found this question on the subject: How do you read from stdin in Python?

A fair way down in the answers is:

The answer proposed by others:

for line in sys.stdin:
  print line

is very simple and pythonic, but it must be noted that the script will wait until EOF before starting to iterate on the lines of input.

This child script behaves as expected:

import sys
import datetime

with open(datetime.datetime.now().strftime('%Y-%m-%d-%H-%M-%S') + '.txt','w') as f:

    line = sys.stdin.readline()
    while line:
        f.write(datetime.datetime.now().isoformat() + ' ' + line)
        line = sys.stdin.readline()

    f.write('Finished')

Upvotes: 1

jfs
jfs

Reputation: 414315

It is the read-ahead bug in Python 2: for line in sys.stdin: doesn't yield anything until its internal buffer is full. Use for line in iter(sys.stdin.readline, ''):, to workaround it.

Upvotes: 1

Related Questions