CODEWITHSUNDEEP
pythonsubprocessmultiprocessingpopen
Ivan Gromov
Ivan Gromov

Reputation: 4425

Background process in Python

I need to create a background process that will wait for incoming commands and perfom them. Here's the code:

instance_tuple.popen = subprocess.Popen(['python',\
                                    os.path.join(config['scripts_dir'],\
                                    'instance_script.py')],\
                                    stdin = subprocess.PIPE,\
                                    stdout = subprocess.PIPE)

Process function code:

if __name__ == '__main__':
    config = dict()
    is_config_valid = False
    print 'Hello from instance process'
    while True:
        cmd_str = raw_input()
        if (cmd_str.strip() != ''):
            print 'received %s' % cmd_str
            command = json.loads(cmd_str)
        print 'received command: %s' % str(command)
        sys.stdout.flush()
        if command['name'] == 'set_variable':
            name = command['args'][0]
            value = command['args'][1]
            config[name] = value
            is_config_valid = validate_instance_dict(config)            
        elif is_config_valid:
            if (command['name'] == 'init_model'):
                config['instance'].init_model()
            elif (command['name'] == 'get_tree'):
                tree = config['instance'].get_fidesys_tree(command['args'])
                result = CommandResult(command.name, tree)
    print 'process exit'

That's how I send data to the process: 1st test run works ok:

(input, errors) = instance_tuple.popen \
                  .communicate(json.dumps({'name': 'name', 'args': list()}))

Later for some reason raw_input() gets a EOF and the process exits. What is the correct way to setup an interprocess communication?

Upvotes: 4

Views: 3950

Answers (4)

driquet
driquet

Reputation: 679

Subprocess module let you do that, but you can't use communicate function.

I like to use pexpect module. It's easier !
Below an example in which a ftp connection is created, and python script interacts with the created process :

 import pexpect
 import sys

 child = pexpect.spawn('ftp ftp.openbsd.org')
 child.expect('name .*: ')
 child.sendline('anonymous')
 child.expect('(password')
 child.sendline('[email protected]')
 child.expect('ftp> ')
 child.sendline('cd /pub/OpenBSD/3.7/packages/i386')
 ...
if child.isalive():
   child.sendline('bye') # Try to ask ftp child to exit.
   child.close()

Upvotes: 0

Macke
Macke

Reputation: 25690

I believe communicate() closes stdin, which gives your process an EOF.

Use popen.stdin.write(...) if you want to talk to it several times.

Upvotes: 0

rocksportrocker
rocksportrocker

Reputation: 7419

What happens if you use "sys.stdin.readline()" instead of raw_input ?

Upvotes: 0

Zach Kelling
Zach Kelling

Reputation: 53869

I like to use zeromq for this. I setup a server using the zmq.PULL socket which listens for clients sending messages using the zmq.PUSH socket. Really easy to work with:

import zmq

def client(msg)
    context = zmq.Context()
    client = context.socket(zmq.PUSH)
    client.connect('tcp://127.0.0.1:9999')
    client.send(msg)

def server():
    context = zmq.Context()
    server = context.socket(zmq.PULL)
    server.bind('tcp://127.0.0.1:9999')

    while True:
        msg = server.recv()
        ..do something with each message

if __name__ == '__main__': server()

Upvotes: 5

Related Questions