CODEWITHSUNDEEP
pythonpython-3.xsubprocesssignals
Erika L
Erika L

Reputation: 307

python subprocess.call doesn't handle signal correctly

(I'm using Python 3.4.2) I have a script test.py, which handles SIGTERM etc. However, when it's called by some other script, the sig-handling wasn't correct.

This is test.py:

#! /path/to/python3
import time
import signal
import sys

def handleSIG(signal, frame):
    for i in range(10):
        print(i)
    sys.exit()

for sig in [signal.SIGTERM, signal.SIGINT, signal.SIGQUIT, signal.SIGHUP]:
    signal.signal(sig, handleSIG)

time.sleep(30)

If I just call "test.py" and do "Ctrl+C", then it prints 0,1,...,9 to the console. However, if I call test.py in another script using subprocess.call, only 0 will be printed. For example, here's another script that calls test.py:

import subprocess

cmd = '/path/to/test.py'
subprocess.call(cmd)

Strangely, using subproces.Popen() makes this error go away.

Upvotes: 3

Views: 5844

Answers (2)

Stuart Berg
Stuart Berg

Reputation: 18141

UPDATE: This Python-3 regression will be fixed in Python 3.7, via PR #5026. For additional background and discussion, see bpo-25942 and (rejected) PR #4283.

I ran into this issue myself recently. The explanation given by @pilcrow is correct.

The OP's solution (in the comments) of merely using the Python 2 implementation (Popen(*popenargs, **kwargs).wait()) doesn't suffice for me, because I'm not 100% sure that the child will respond to SIGINT in all cases. I still want it to be killed -- after a timeout.

I settled on simply re-waiting for the child (with timeout).

def nice_call(*popenargs, timeout=None, **kwargs):
    """
    Like subprocess.call(), but give the child process time to
    clean up and communicate if a KeyboardInterrupt is raised.
    """
    with Popen(*popenargs, **kwargs) as p:
        try:
            return p.wait(timeout=timeout)
        except KeyboardInterrupt:
            if not timeout:
                timeout = 0.5
            # Wait again, now that the child has received SIGINT, too.
            p.wait(timeout=timeout)
            raise
        except:
            p.kill()
            p.wait()
            raise

Technically, this means that I'm potentially extending the life of the child beyond the original timeout, but that's better than incorrect cleanup behavior.

Upvotes: 2

pilcrow
pilcrow

Reputation: 58534

The python 3.3 subprocess.call implementation sends a SIGKILL to its child if its wait is interrupted, which it is by your Ctrl-C (SIGINT -> KeyboardInterrupt exception).

So, you see a race between the child process handling the terminal's SIGINT (sent to the whole process group) and the parent's SIGKILL.

From the python 3.3 sources, edited for brevity:

def call(*popenargs, timeout=None, **kwargs):
    with Popen(*popenargs, **kwargs) as p:
        try:
            return p.wait(timeout=timeout)
        except:
            p.kill()
            p.wait()
            raise

Contrast this with the python 2 implementation:

def call(*popenargs, **kwargs):
    return Popen(*popenargs, **kwargs).wait()

What an unpleasant surprise. It appears that this behavior was introduced in 3.3 when the wait and call interfaces were extended to accommodate a timeout. I don't find this correct, and I've filed a bug.

Upvotes: 8

Related Questions