CODEWITHSUNDEEP
pythonsubprocess
Mustafa Quraish
Mustafa Quraish

Reputation: 694

subprocess.call() not killing the process on timeout

I'm not going to do into details of why i need to do this, but my problem is as follows: I have a string containing some shell commands I need to run, which might have some output redirections in them. Consider the simple example:

cmd = "yes > output.txt"

I want to be able to run this command through python with a timeout so the command doesn't run forever if there's an infinite loop. I'm using the subprocess module. Here is my code:

import subprocess

subprocess.call("yes > output.txt", timeout=1, shell=True)

When I run this, I get a subprocess.TimeoutExpired exception as expected after 1 second, but the process is not killed. If I look at htop I still see the yes process running.

Curiously, if I run this through the interpreter, and I press Ctrl+C at the prompt after the exception was thrown, that kills the process. Am I just doing something dumb here?

I get this behaviour both on macOS Catalina with Python 3.7.7 as well as on Ubuntu 18.04 with Python 3.6.9

EDIT:

I'm unsure why there's an inconsistency if my command redirects output to a file. For example:

subprocess.call("sleep 100", timeout=1, shell=True)

does kill the process when it times out. However, the following:

subprocess.call("sleep 100 > f", timeout=1, shell=True)

does not kill the process. I'm aware that in this case nothing is actually redirected to the file.

More importantly, my actual question is, how would i go about killing the child process in all cases after timeout?

Upvotes: 1

Views: 2498

Answers (1)

Serge Ballesta
Serge Ballesta

Reputation: 148890

The timeout parameter is not the time allowed to the child process, but just the time allowed to the parent to wait for the end of its child. If the child does not end during the allowed time, nothing happens to the child and the parent receives a TimeoutExpired exception. So what you observe is by design.

When you type Ctrl C in a interactive terminal session, the SIGINT interruption is sent to all the processes in the process group of the foreground process. As you have not explicitely started a new process group, childs of your Python process will share the same process group and will receive the SIGINT. Here again, what you observe is by design.

The sleep program is special. It internally depends on the SIGALARM interruption, as does subprocess.call with the timeout parameter. Here again, the signal is sent to the whole process group, hence the child terminates. But I have no explainations on why redirecting output to a file prevents the child to be killed.

If you want the child to be forcibly terminated, you should not use call but create and manage explicitely a Process object:

p = subprocess.Process("yes > output.txt", shell=True)
try:
    p.wait(timeout=1)
except subprocess.TimeoutExpired:
    p.terminate()

Beware untested code, so typos could be present...

Upvotes: 2

Related Questions