CODEWITHSUNDEEP
pythonstreamsubprocesspipe
Alejandro Garcia
Alejandro Garcia

Reputation: 1221

Using python Popen to read the last line

I have a simple python program:

test.py:

import time
for i in range(100000):
    print i
    time.sleep(0.5)

I want to use another program that executes the above one in order to read the last line output while the above program is counting.

import subprocess

process = subprocess.Popen("test",stdout=PIPE)
sleep(20) # sleeps an arbitrary time
print stdout.readlines()[-1]

The problem is that process.stdout.readlines() waits until test.py finishes execution. Is there any way to read the last line that has been writen in the output while the program is executing?

Upvotes: 5

Views: 8514

Answers (2)

amoffat
amoffat

Reputation: 714

Fairly trivial with sh.py:

import sh

def process_line(line):
    print line

process = sh.python("test.py", _out=process_line)
process.wait()

Upvotes: 1

jfs
jfs

Reputation: 414715

You could use collections.deque to save only the last specified number of lines:

#!/usr/bin/env python
import collections
import subprocess
import time
import threading

def read_output(process, append):
    for line in iter(process.stdout.readline, ""):
        append(line)

def main():
    process = subprocess.Popen(["program"], stdout=subprocess.PIPE)
    # save last `number_of_lines` lines of the process output
    number_of_lines = 1
    q = collections.deque(maxlen=number_of_lines)
    t = threading.Thread(target=read_output, args=(process, q.append))
    t.daemon = True
    t.start()
    #
    time.sleep(20)

    # print saved lines
    print ''.join(q),
    # process is still running
    # uncomment if you don't want to wait for the process to complete
    ##process.terminate() # if it doesn't terminate; use process.kill()
    process.wait()

if __name__=="__main__":
    main()

See other tail-like solutions that print only the portion of the output

See here if your child program uses a block-buffering (instead of line-bufferring) for its stdout while running non-interactively.

Upvotes: 4

Related Questions