CODEWITHSUNDEEP
pythonsubprocessfork
Prakash Raman
Prakash Raman

Reputation: 13923

Python create a subprocess and do not wait

I would like to run a series of commands (which take a long time). But I do not want to wait for the completion of each command. How can I go about this in Python?

I looked at 

os.fork()

and 

subprocess.popen()

Don't think that is what I need.

Code

def command1():
   wait(10)

def command2():
   wait(10)

def command3():
   wait(10)

I would like to call

command1()
command2()
command3()

Without having to wait.

Upvotes: 1

Views: 4218

Answers (3)

ValeriyKr
ValeriyKr

Reputation: 131

This example maybe is suitable for you:

#!/usr/bin/env python3

import sys
import os
import time

def forked(fork_func):
    def do_fork():
        pid = os.fork()
        if (pid > 0): 
            fork_func()
            exit(0)
        else:
            return pid
    return do_fork

@forked
def command1():
    time.sleep(2)

@forked
def command2():
    time.sleep(1)

command1()
command2()
print("Hello")

You just use decorator @forked for your functions.

There is only one problem: when main program is over, it waits for end of child processes.

Upvotes: 2

jsbueno
jsbueno

Reputation: 110166

The most straightforward way is to use Python's own multiprocessing:

from multiprocessing import Process

def command1():
   wait(10)
...

call1 = Process(target=command1, args=(...))
call1.start()
...

This module was introduced back exactly to ease the burden on controlling external process execution of functions accessible in the same code-base Of course, that could already be done by using os.fork, subprocess. Multiprocessing emulates as far as possible, Python's own threading moudle interface. The one immediate advantage of using multiprocessing over threading is that this enables the various worker processes to make use of different CPU cores, actually working in parallel - while threading, effectively, due to language design limitations is actually limited to a single execution worker at once, thus making use of a single core even when several are available.

Now, note that there are still peculiarities - specially if you are, for example, calling these from inside a web-request. Check this question an answers form a few days ago: Stop a background process in flask without creating zombie processes

Upvotes: 1

Manish Gupta
Manish Gupta

Reputation: 4656

Use python's multiprocessing module.

def func(arg1):
    ... do something ...

from multiprocessing import Process
p = Process(target=func, args=(arg1,), name='func')
p.start()

Complete Documentaion is over here: https://docs.python.org/2/library/multiprocessing.html

EDIT:

You can also use the Threading module of python if you are using jpython/cpython distribution as you can overcome the GIL (Global Interpreter Lock) in these distributions.

https://docs.python.org/2/library/threading.html

Upvotes: 3

Related Questions