CODEWITHSUNDEEP
pythonpython-3.xsubprocess
Tom Ritter
Tom Ritter

Reputation: 101330

Is there a way to both stream and capture output from subprocess.run in Python 3?

If you do

subprocess.run(["echo hi && sleep 60"], shell=True, capture_output=False, timeout=5)

you will see 'hi' printed to the terminal and then an exception will be thrown due to the timeout.

But if you do

ret = subprocess.run(["echo hi && sleep 6"], capture_output=True, timeout=2, shell=True)

You won't see any output, and the timeout exception will (as far as I can tell) prevent you from getting the output that was produced. (The shell=True is not important for this example, I just needed a convenient way to make it timeout.)

This is causing issues where I (a) need to get the output of the command but (b) sometimes am inadvertently running commands that hang waiting for user input. I'd really like to be able to capture any output produced despite a Timeout Exception being thrown or stream the output to stdout/stderr while also getting a copy of it.

Upvotes: 1

Views: 318

Answers (1)

ShadowRanger
ShadowRanger

Reputation: 155363

The TimeoutExpired exception itself includes any captured output as attributes (stdout and stderr), so:

try:
    ret = subprocess.run("commands go here && sleep 6", capture_output=True, timeout=2, shell=True)
except subprocess.TimeoutExpired as e:
    print(e.stdout)
else:
    print(ret.stdout)

will work when the command in question actually writes to the pipe prior to the timeout.

But if the subprocess was killed before completion, while it still had data in user-mode buffers (it was block buffering, or line buffering and never got around to writing a newline or whatever), it would never have been written to the actual pipe, so you won't see it.

Unbuffering the child process is done in many different ways, so I can't give specific info on how to do it in your real case (echo is obviously just a placeholder), the stdbuf is often useful when the child executable can't be modified to make it line-buffered or unbuffered.

Upvotes: 1

Related Questions