CODEWITHSUNDEEP
pythonsubprocess
borges
borges

Reputation: 3687

Run and get output from a background process

I already know there is several questions targeting this theme, but none of them solved my specific problem. Or at least I could not find it.

I need to execute some program in background, wait for output and manipulate it. But the background program must keep executing.

The info from the background program that I need is located, precisely, in the second line of its output. There is no problem if this program blocks my code until reach this line. But it is essential that it unlocks after that line so I can perform other tasks completely unrelated with the background program.

Still, I cannot figure out how to accomplish this. I've read a lot the documentation of subprocess module, in particular, subprocess.Popen.

Being pratical: why this code does not work with ['localtunnel', '8000'] argument? It outputs nothing...

I know I do not need root privileges to execute this.

EDIT after answers from jadkik94 and fest

Unfortunately, both answers do not work for me. Maybe I am doing something wrong...

First of all. A 'sanity check':

import subprocess, threading, time
can_break = False

def run():
    args = ["ping", "google.com"]
    popen = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE)
    while not can_break:
        print popen.stdout.readline()

t = threading.Thread(target=run)

try:
    t.start()
    while True:
        print 'Main thread...'
        time.sleep(1)
except KeyboardInterrupt:
    can_break = True

The code above works properly with output similar to this:

Main thread...
PING google.com (74.125.234.160) 56(84) bytes of data.
64 bytes from plusone.google.com (74.125.234.160): icmp_req=1 ttl=54 time=82.5 ms
Main thread...
64 bytes from plusone.google.com (74.125.234.160): icmp_req=2 ttl=54 time=82.7 ms
[...]

But when I use it with the args I want (args = ['localtunnel', 8000]), the only output is Main thread....

When I call localtunnel in the main thread (blocking), it returns the desired output:

In [x]: popen = subprocess.Popen(['localtunnel', '8000'])
  This localtunnel service is brought to you by Twilio.
  Port 8000 is now publicly accessible from http://????.localtunnel.com ...

This approach is based on jadkik94's answer. But fest's answer does not work either.

Upvotes: 3

Views: 17317

Answers (2)

jadkik94
jadkik94

Reputation: 7078

You can run it in a thread (so that it doesn't block your code from running), and get the output until you get the second line, then wait for it to terminate. This is an example that will read the output from the command dir /s on Windows to get all the directory listing.

import subprocess, thread, time

def run():
    global can_break

    args = ["dir", "/s"]
    shell = True

    count = 0
    popen = subprocess.Popen(args, shell=shell, stdout=subprocess.PIPE)
    while True:
        line = popen.stdout.readline()
        if line == "": continue
        count += 1
        if count == 2:
            do_something_with(line)
            break

    print "We got our line, we are waiting now"
    popen.wait()
    print "Done."
    can_break = True

def do_something_with(line):
    print '>>> This is it:', line

thread.start_new_thread(run, tuple())

can_break = False
while not can_break:
    print 'Wait'
    time.sleep(1)

print 'Okay!'

Output will look like:

Wait
>>> This is it:  Volume Serial Number is XXXX-XXXX

We got our line, we are waiting now
Wait
Wait
Wait
.
.
.
Done.
Wait
Okay!

Upvotes: 0

fest
fest

Reputation: 1635

To launch a program in a non blocking way but still being able to see the output of program, the program has to be launched in a separate thread or process. Ryan has posted a nice code sample here: Python Subprocess.Popen from a thread

Keep in mind that the last line print myclass.stdout will print the output how it appears at that time. If program is just being launched, it might not have output anything at all, so your code should probably read from myclass.stdout until it receives the line you need.

Upvotes: 1

Related Questions