CODEWITHSUNDEEP
pythonwindowssubprocesspopen
khattam
khattam

Reputation: 1174

Popen waiting for child process even when the immediate child has terminated

I'm working with Python 2.7 on Windows 8/XP.

I have a program A that runs another program B using the following code:

p = Popen(["B"], stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
return

B runs a batch script C. C is a long running script and I want B to exit even though C has not finished. I have done it using the following code (in B):

p = Popen(["C"])
return

When I run B, it works as expected. When I run A however, I expected it to exit when B exits. But A waits until C exits even though B has already exitted. Any ideas on what's happening and what possible solutions could be?

Unfortunately, the obvious solution of changing A to look like B is not an option.

Here is a functional sample code to illustrate this issue: https://www.dropbox.com/s/cbplwjpmydogvu2/popen.zip?dl=1

The zip file consists of the following files with the following contents:

A.py

from subprocess import PIPE, Popen
import sys

def log(line):
    with open("log.txt", "a") as logfile:
        logfile.write(line)

log("\r\n\r\nA: I'll wait for B\r\n")

p = Popen(["C:\\Python27\\python.exe", "B.py"], stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()

log("A: Done.\r\n")
sys.exit(0)

B.py

from subprocess import Popen, PIPE
import sys

def log(line):
    with open("log.txt", "a") as logfile:
        logfile.write(line)

log("B: launching C\r\n")

p = Popen(["C.bat"])

log("B: Not waiting for C at all. bye!\r\n")
sys.exit(0)

C.bat

@echo off
echo C: Start long running task : %time% >>  "log.txt"
ping -n 10 127.0.0.1>nul
echo C: Stop long running task : %time% >>  "log.txt"

Any input is much appreciated.

Upvotes: 22

Views: 14279

Answers (3)

Robert
Robert

Reputation: 1631

An alternative would be to start C as fully forked-off process in a separate process tree, e.g. via the start command of cmd.exe:

import subprocess
subprocess.Popen(["cmd.exe", "/C", "start notepad"])

Obviously, since it is completely independent, you cannot communicate with it. But you can use psutil to retrieve it's PID to at least monitor or close it, if necessary:

import psutil
for process in psutil.process_iter():
    if process.name() == 'notepad.exe':
        print(process.pid)

Upvotes: 0

jtpereyda
jtpereyda

Reputation: 7385

Here is a code snippet adapted from Sebastian's answer and this answer:

#!/usr/bin/env python
import os
import sys
import platform
from subprocess import Popen, PIPE

# set system/version dependent "start_new_session" analogs
kwargs = {}
if platform.system() == 'Windows':
    # from msdn [1]
    CREATE_NEW_PROCESS_GROUP = 0x00000200  # note: could get it from subprocess
    DETACHED_PROCESS = 0x00000008          # 0x8 | 0x200 == 0x208
    kwargs.update(creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, close_fds=True)  
elif sys.version_info < (3, 2):  # assume posix
    kwargs.update(preexec_fn=os.setsid)
else:  # Python 3.2+ and Unix
    kwargs.update(start_new_session=True)

p = Popen(["C"], stdin=PIPE, stdout=PIPE, stderr=PIPE, **kwargs)
assert not p.poll()

I've only tested it personally on Windows.

Upvotes: -1

jfs
jfs

Reputation: 414079

You could provide start_new_session analog for the C subprocess:

#!/usr/bin/env python
import os
import sys
import platform
from subprocess import Popen, PIPE

# set system/version dependent "start_new_session" analogs
kwargs = {}
if platform.system() == 'Windows':
    # from msdn [1]
    CREATE_NEW_PROCESS_GROUP = 0x00000200  # note: could get it from subprocess
    DETACHED_PROCESS = 0x00000008          # 0x8 | 0x200 == 0x208
    kwargs.update(creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP)  
elif sys.version_info < (3, 2):  # assume posix
    kwargs.update(preexec_fn=os.setsid)
else:  # Python 3.2+ and Unix
    kwargs.update(start_new_session=True)

p = Popen(["C"], stdin=PIPE, stdout=PIPE, stderr=PIPE, **kwargs)
assert not p.poll()

[1]: Process Creation Flags for CreateProcess()

Upvotes: 31

Related Questions