CODEWITHSUNDEEP
pythonsubprocess
nerdinary
nerdinary

Reputation: 67

How to stream subprocess output with a user prompt?

Here's my subprocess call:

def myrun(cmd):
  p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  stdout = []
    while True:
      line = p.stdout.readline()
      stdout.append(line)
      print line,
      if line == '' and p.poll() != None:
          break
  return ''.join(stdout)

When I run the 'cmd' normally, the output is usually something like:

some text...
some more text...
Do you want to continue [yes/no]? : y
more output...

But running the same 'cmd' with subprocess as shown above, my output is like this:

Do you want to continue [yes/no]? : y
some text...
some more text...
more output...

How do I fix this?

Upvotes: 2

Views: 774

Answers (1)

thiruvenkadam
thiruvenkadam

Reputation: 4250

This is basically because of the buffering that is there normally at whatever your cmd program has. You have to disable the default buffering happening at that program in order to attain what you are looking for.

In case it is a python file you are running through the cmd like the following, you can add "-u" parameter with the python command to get your desired result

cmd = ["python", "-u", "/link/to/python/file.py"]
myrun(cmd)

As per the man docs of python, 

-u     : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x

In case of any file that is to run in the shell, I would say stdbuf command is your safe bet. Set the -o option to 0(zero) along with the command and you are good to go.

cmd = ["sh", "\path\to\sh\file.sh"]
unbuffered_cmd = ["stdbuf", "-o0", "-e0"] + cmd
myrun(unbuffered_cmd)

Please note that stdbuf comes with GNU coreutils. So it may not be platform agnostic.

To be on the safer side even with stdbuf present and "-u" set for python, set the bufsize parameter of Popen to 1

Popen(cmd, stdout=PIPE, stderr=STDOUT, bufsize=1)

I am out of options, if stdbuf is also not helping :-)

Upvotes: 2

Related Questions