CODEWITHSUNDEEP
pythonsubprocess
WesR
WesR

Reputation: 1512

python subprocess.popen kill subprog when parent prog exits?

In python 2.7 if program master uses subprocess.Popen to start subprocess and then exits, subprocess will continue to run. I think that this is consistent with process handling in general, at least for *nix.

This seems to imply that if some of my subprocess programs run forever and I kill master the subprocesses will continue to run. Is that correct? (I ask because of the possibility that killing a process behaves differently than simply having it exit of its own accord).

In StackOverflow entries about command shells (not Python) there is mention of being able to create a subprocess that actually will terminate if its creator process dies. They seem to be called "daemonic" processes.

Is there a similar way to make this happen automatically in Python? That is, is there a way I can use subprocess.Popen or another method to start a subprocess that will automatically die if the parent dies?

The child processes are very different so threading is not an option.

Upvotes: 1

Views: 2141

Answers (1)

abarnert
abarnert

Reputation: 366013

The term "daemon" is probably more confusing than helpful here. It means something you don't have to wait/join. For a thread, that actually means a thread that will be implicitly killed on exit—but for a child process, it means one that will run in the background and continue to do so even if its parent, its session, its tty, etc. all die. The subprocess docs don't even mention the term, possibly to avoid this confusion.

At any rate, if you want children that automatically go away with the parent no matter what on all platforms, you need to do that explicitly.

For a nice design where there's a top-level script that has access to all the Popen object, the easy way to do that is at the top level:

if __name__ == '__main__':
    try:
        # do a bunch of stuff, including launching lots of children
    finally:
        for p in childprocs:
            p.kill()
        for p in childprocs:
            p.wait()

You can of course get fancier by waiting with a timeout and then terminating, or just terminate them without trying to kill first, or send different signals, or… whatever you want. But this is the basic idea.

In some cases, it's even nicer to use a with statement with an ExitStack that you add the children to as they're created.

On the other hand, in some complex scripts, there's really no good way to access the children from the top level, and you have to use something cruftier like atexit handlers.

But there's another simple design that often works. If you're not running too many children, you can spawn a thread just to wait on each one, or even to just run them instead of creating a Popen. If you are running lots of children, a single thread can loop through them waiting with a timeout, although this can be a lot more fiddly to get right (and waste resources if you don't). Then it's just a matter of managing your threads, and they automatically manage your children.

Upvotes: 1

Related Questions