CODEWITHSUNDEEP
pythonlinuxsubprocess
Sergey
Sergey

Reputation: 12417

"Fire and forget" a process from a Python script

How do I start a process (another Python script, for example) from a Python script so the "child" process is completely detached from the "parent", so the parent can a) continue on its merry way without waiting for child to finish and b) can be terminated without terminating the child process?

Parent:

import os

print "Parent started"
os.system("./child.py")
print "Parent finished"

Child:

import time

print "Child started"
time.sleep(10)
print "Child finished"

Running parent.py prints:

Parent started
Child started
Child finished
Parent finished

What I want it to print:

Parent started
Child started
Parent finished
(seconds later)
Child finished

Upvotes: 16

Views: 10284

Answers (3)

Sergey
Sergey

Reputation: 12417

Answering my own question: I ended up simply using os.system with & at the end of command as suggested by @kevinsa. This allows the parent process to be terminated without the child being terminated.

Here's some code:

child.py

#!/usr/bin/python

import time
print "Child started"
time.sleep(10)
print "Child finished"

parent.py, using subprocess.Popen:

#!/usr/bin/python

import subprocess
import time

print "Parent started"
subprocess.Popen("./child.py")
print "(child started, sleeping)"

time.sleep(5)

print "Parent finished"

Output:

$ ./parent.py
Parent started
(child started, sleeping)
Child started
^CTraceback (most recent call last):
Traceback (most recent call last):
  File "./child.py", line 5, in <module>
  File "./parent.py", line 13, in <module>
        time.sleep(10)
time.sleep(5)
KeyboardInterrupt
KeyboardInterrupt
  • note how the child never finishes if the parent is interrupted with Ctrl-C

parent.py, using os.system and &

#!/usr/bin/python

import os
import time

print "Parent started"
os.system("./child.py &")
print "(child started, sleeping)"

time.sleep(5)

print "Parent finished"

Output:

$ ./parent.py
Parent started
(child started, sleeping)
Child started
^CTraceback (most recent call last):
  File "./parent.py", line 12, in <module>
    time.sleep(5)
KeyboardInterrupt

$ Child finished

Note how the child lives beyond the Ctrl-C.

Upvotes: 1

nehem
nehem

Reputation: 13642

Using asyncio you can write a simple decorator as @background

import asyncio
import time

def background(f):
    def wrapped(*args, **kwargs):
        return asyncio.get_event_loop().run_in_executor(None, f, *args, *kwargs)

    return wrapped

@background
def foo():
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")

Produces

>>> Hello
>>> I didn't wait for foo()
>>> foo() completed

Upvotes: 4

RubenLaguna
RubenLaguna

Reputation: 24776

Since you mentioned os.system, I think it's worth to mention that you should have used os.spawn* with mode P_NOWAIT to achieve the "forget" part.

But subprocess module provides replacements for os.system, os,spawn*,etc so you should use that instead like so

import subprocess
p = subprocess.Popen("./child.py")
print "pid = ", p.pid

See Replacing os.spawn with subprocess.Popen

As I explained in the comments both processes parent.py and child.py are still on the same process group and therefore the terminal will forward signals (like Ctrl-C) to all process in the foreground process group so both will get killed when you Ctrl-C. So if you don't want that you can force child.py to be in a new process group with the following: 

#!/usr/bin/env python
import subprocess
import time
import os
p = subprocess.Popen("./child.py", preexec_fn=os.setsid)
print "pid = ", p.pid
time.sleep(30) # Ctrl-C at this point will not kill child.py
print "parent exit"

Upvotes: 12

Related Questions