CODEWITHSUNDEEP
pythonsubprocessstdoutexecute
Nakiscia
Nakiscia

Reputation: 20

Executing external Python file in background and get output in python

I have an python file that takes some inputs and print outputs according to these inputs. I am try to execute this script in another script. All I need to do is sending some parameters to this file and getting the outputs in this script.

The script that I need to be executed : 

while True:
  print("Sabah 1")
  print("Oglen 2")
  print("Aksam 3")
  print("Gece 4")
  print("---------")


  secim = raw_input("Gun icerisindeki zaman dilimini giriniz")

  isim = raw_input("Isminizi giriniz.")


  if (secim=='1') : 
      print("Gunaydin"+isim)
  elif (secim == '2'):
      print("Tunaydin"+isim)
  elif (secim == '3'):
      print("iyi aksamlar"+isim)
  elif (secim == '4'):
      print("Iyi geceler"+isim)
  else:
      print("Program sonlandiriliyor")
      break

The script that should execute the script above : 

import subprocess, threading, time
can_break = False

def run():
    args = ['python','odev2.py','arg1','arg2']
    popen = subprocess.Popen(args,shell=False,stdout=subprocess.PIPE)
    while not can_break:
        print(popen.stdout.readline())


t = threading.Thread(target=run)

try:
    t.start()
    while True: 
        print('Main Thread.... ')
        time.sleep(1)
except KeyboardInterrupt:
    can_break_break = True

The problem is the output of the second script keep printing 'Main Thread...' I cannot read any output.

Thanks.

Upvotes: -1

Views: 749

Answers (1)

Peter Moore
Peter Moore

Reputation: 86

I assume you are wondering why you aren't seeing any of the prompts from odev2.py?

Short answer is to replace run with:

def run():
    args = ['python','-u','odev2.py','arg1','arg2']
    popen = subprocess.Popen(args,shell=False,stdout=subprocess.PIPE)
    while not can_break:
        sys.stdout.write(popen.stdout.read(1))

The long answer is that there are two issues.

First, there is buffering. IO is expensive, so rather than immediately writing out your data, File objects will sometimes instead collect the data in a buffer and only write the data out when the buffer is full. The builtin file objects (like sys.stdout) will decide to buffer depending on what they are writing to. If they are writing to a terminal, then they will turn off buffering so the output gets to the user immediately. But if they are writing to a file, they will buffer to make it more efficient.

So when you run odev2.py at the command line, print is writing to a terminal and so it is unbuffered and your prompts appear immediately. But when you run odev2.py with subprocess(stdout=PIPE), odev2.py is getting a pipe as standard output so the file object buffers. That means you don't see the prompt.

You can see the same effect running odev2.py from the command line using 'python odev2.py arg1 arg2 | cat'. This sets standard output to a pipe and you won't see any prompt, but odev2.py will be waiting for you to type.

This buffering problem can be fixed by having odev2.py write the prompts itself (rather than having raw_input print them) and then call sys.stdout.flush() at the appropriate times. But a quick and dirty way is to run python with the -u flag that turns off all buffering. Try 'python -u odev2.py arg1 arg2 | cat' to see the difference.

Note: -u turns off all buffering, which can have a huge impact on performance. So this is only a quick and dirty solution

Second, your reading thread uses readline. readline won't return until it sees a newline, but odev2.py's prompts don't end with newlines. So the readline in run won't return until odev2.py sends a newline, but odev2 won't send a newline until it reads some input from you. So you are deadlocked.

The robust solution for this is to replace readline with a non-blocking IO that will return you what is available. But there doesn't seem to be a portable way to do that in python 2.7. So quick hack is to do single character reads. So instead of doing popen.stdout.readline(), you just do popen.stdout.read(1).

Upvotes: 1

Related Questions