CODEWITHSUNDEEP
pythonsubprocess
Nemelis
Nemelis

Reputation: 5462

Output of subprocess both to PIPE and directly to stdout

I found a number of questions which looks like mine, but which did not produce a solution I can use (closest is: subprocess output to stdout and to PIPE)

The problem: I want to start a process using subprocess which takes a long time. After running the command I need to parse the stdout-output and the stderr-output.

Currently I do it as follows:

p = subprocess.Popen( command_list, stdout=subprocess.PIPE, 
    stderr=subprocess.PIPE )
out, error_msg = p.communicate()
print out + "\n\n" + error_msg

#next comes code in which I check out and error_msg

But the drawback of this method is that the user does not see the output of the process while it is running. Only at the end the output is printed.

Is there a way that the output is printed while the command is running (as if I gave the command without stdout/stderr=subprocess.PIPE) and still have the output via p.communicate in the end?

Note: I'm currently developing on python 2.5 (old software release which uses this python version).

Upvotes: 13

Views: 8471

Answers (2)

Noam Manos
Noam Manos

Reputation: 16971

Here's a python function that watches the command stdout + stderr, and returns the output if finished successfully, else raises an exception:

import subprocess
def execute_command(cmd):
    """
    Execute and watch shell command, and return its output
    """
    output = ''
    with subprocess.Popen(
            cmd,
            shell=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            universal_newlines=True,
            encoding='utf-8',
            errors='replace'
        ) as p:
            while True:
                line = p.stdout.readline()
                if line != '':
                    output += (line + "\n")
                    print(line)
                elif p.poll() != None:
                    break
            sys.stdout.flush()
            if (p.returncode == 0):
                return output
            else:
                raise Exception(cmd, p.returncode, output)

Upvotes: 3

Scis
Scis

Reputation: 2984

This snippet has helped me once in a similar situation:

process = subprocess.Popen(cmd, bufsize=1, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(process.stdout.readline, ''):
    print line,
    sys.stdout.flush() # please see comments regarding the necessity of this line 
process.wait()
errcode = process.returncode

Upvotes: 8

Related Questions