CODEWITHSUNDEEP
pythonsubprocess
beary605
beary605

Reputation: 824

How to implement Streaming Input with Subprocesses in PYTHON?

Since input and raw_input() stop the program from running anymore, I want to use a subprocess to run this program...

while True: print raw_input()

and get its output.

This is what I have as my reading program:

import subprocess
process = subprocess.Popen('python subinput.py', stdout=subprocess.PIPE, stderr=subprocess.PIPE) 
while True:
    output=process.stdout.read(12)
    if output=='' and process.poll()!=None:
        break
    if output!='':
        sys.stdout.write(output)
        sys.stdout.flush()

When I run this, the subprocess exits almost as fast as it started. How can I fix this?

Upvotes: 0

Views: 628

Answers (3)

Tomasz Gandor
Tomasz Gandor

Reputation: 8833

I'm afraid it won't work this way.

You assume, that subprocess will attach your console (your special case of stdin). This does not work, the module only has two options for specifying that: PIPE and STDOUT.

When nothing is specified, the subprocess won't be able to use the corresponding stream - it's output will go nowhere or it will receive no input. The raw_input() ends because of EOF.

The way to go is to have your input in the "main" program, and the work done in a subprocess.

EDIT:

Here's an example in multiprocessing

from multiprocessing import Process, Pipe
import time

def child(conn):
    while True:
        print "Processing..."
        time.sleep(1)
        if conn.poll(0):
            output = conn.recv()
            print output
        else:
            print "I got nothing this time"

def parent():
    parent_conn, child_conn = Pipe()
    p = Process(target=child, args=(child_conn,))
    p.start()
    while True:
        data = raw_input()
        parent_conn.send(data)
    # p.join() - you have to find some way to stop all this...
    # like a specific message to quit etc.


if __name__ == '__main__':
    parent()

You of course need to make it more robust by finding a way too stop this cooperation. In my example both processes are in the same file, but you may organize it differently.

This example works on Linux, you may have some problems with pipes on Windows, but it should be altogether solvable.

The "Processing" is the part where you want to do something else, not just wait for the data from the parent.

Upvotes: 1

Kracekumar
Kracekumar

Reputation: 20419

Well, try different architecture. You can use zeromq.

  1. Producer produces all the items(here output which to be sent via stdout) and broadcasted via zmq.

  2. Consumer should listen to the port no which is being broadcasted by the producer and process them accordingly.

Here is the Example http://code.saghul.net/implementing-a-pubsub-based-application-with

Note

Use gevent or multiprocessing to spawn these process.

You will have master program which takes care of spawning producer and consumer

Upvotes: 0

Andrew Gorcester
Andrew Gorcester

Reputation: 19963

I think the problem is that subprocesses are not directly hooked up to stdout and stdin, and therefore cannot receive keyboard input. Presumably raw_input() is throwing an exception.

If this is a practical issue and not an experiment, I recommend you use a library such as curses or pygame to handle your input. If you're experimenting and want to do it yourself, then I suppose you'll have to look at threads instead of subprocesses, though this is a fairly complex thing to try to do so you're certain to run into other issues.

Upvotes: 1

Related Questions