CODEWITHSUNDEEP
pythonlinuxbash
Alex
Alex

Reputation: 44305

How to execute a bash script from within python in a non-blocking way, and allowing to see its output?

On a Ubuntu Linux system I want to start a bash script from within a python script. I need to run it in a non-blocking way so that the python script reports all output from the bash script to the standard output.

I have found a very similar question here which does not work for me (of course). Here is my bash script testbash.sh which just prints a row of numbers:

#!/bin/bash
i=0

while [ $i -le 10 ]
do
  echo Number: $i
  ((i++))
  sleep 1
done

Here is the python script:

import sys
import time
from subprocess import PIPE, Popen
from threading  import Thread

try:
    from queue import Queue, Empty
except ImportError:
    from Queue import Queue, Empty  # python 2.x


ON_POSIX = 'posix' in sys.builtin_module_names

def enqueue_output(out, queue):
    for line in iter(out.readline, b''):
        queue.put(line)
    out.close()

p = Popen(['.','testbash.sh'], stdout=PIPE, bufsize=1, close_fds=ON_POSIX, shell=True)
q = Queue()
t = Thread(target=enqueue_output, args=(p.stdout, q))
t.daemon = True # thread dies with the program
t.start()

# read line without blocking
running = True
while running:
    time.sleep(1)
    try:
        line = q.get_nowait() # or q.get(timeout=.1)
    except Empty:
        print('no output yet')
    else:
        # got line
        print(line)

When running this script, I only get the output 'no output yet' and no actual output from the bash script.

I want the python script to run as long as the bash script runs, redirecting the output from the bash script to the python standard-output (so that I see the numbers 1..10 in like every 1-2 seconds), and after ~10 seconds the python script stops. I also need to check if the bash script did finish fine or with an error

What am I missing? Is the script actually running? How to get its output? How to check the exit code?

Upvotes: 0

Views: 658

Answers (1)

Philippe
Philippe

Reputation: 26452

A few changes need to be made, in particular, the calling of shell script:

import sys
import time
from subprocess import PIPE, Popen
from threading  import Thread

try:
    from queue import Queue, Empty
except ImportError:
    from Queue import Queue, Empty  # python 2.x


ON_POSIX = 'posix' in sys.builtin_module_names

def enqueue_output(out, queue):
    for line in iter(out.readline, b''):
        queue.put(line.decode('ascii'))
    out.close()

p = Popen(['./testbash.sh'], stdout=PIPE, bufsize=1, close_fds=ON_POSIX, shell=True)
q = Queue()
t = Thread(target=enqueue_output, args=(p.stdout, q))
t.daemon = True # thread dies with the program
t.start()

# read line without blocking
while t.is_alive():
    time.sleep(1)
    try:
        line = q.get_nowait() # or q.get(timeout=.1)
    except Empty:
        print('no output')
    else:
        # got line
        print(line, end='')

p.wait()
print(f'returncode = {p.returncode}')

Upvotes: 1

Related Questions