How do I kill background processes / jobs when my shell script exits?

Question

I am looking for a way to clean up the mess when my top-level script exits.

Especially if I want to use set -e, I wish the background process would die when the script exits.

Answer 1

This works for me (collaborative effort with the commenters):

trap "trap - SIGTERM && kill -- -$$" SIGINT SIGTERM EXIT

• kill -- -$$ sends a SIGTERM to the whole process group, thus killing also descendants. The <PGID> in kill -- -<PGID> is the group process id, which often, but not necessarily, is the PID that $$ variable contains. The few times PGID and PID differ you can use ps and other similar tools you can obtain the PGID, in your script.

  For example: pgid="$(ps -o pgid= $$ | grep -o '[0-9]*')" stores PGID in $pgid.

• Specifying signal EXIT is useful when using set -e (more details here).

Answer 2

Another option is it to have the script set itself as the process group leader, and trap a killpg on your process group on exit.

EDIT: a possible bash hack to create a new process group is to use setsid(1) but only if we're not already the process group leader (can query it with ps).

Placing this at the beginning of the script can achieve that.

# Create a process group and exec the script as its leader if necessary
[[ "$(ps -o pgid= $$)" -eq "$$" ]] || exec setsid /bin/bash "$0" "$@"

Then signaling the process group with kill -- -$$ would work as expected even when script is not already the process group leader.

Answer 3

None of the answers here worked for me in the case of a continuous integration (CI) script that starts background processes from subshells. For example:

(cd packages/server && npm start &)

The subshell terminates after starting the background process, which therefore ends up with parent PID 1.

With PPID not an option, the only portable (Linux and MacOS) and generic (independent of process name, listening ports, etc.) approach left is the process group (PGID). However, I can't just kill that because it would kill the script process, which would fail the CI job.

# Terminate the given process group, excluding this process. Allows 2 seconds
# for graceful termination before killing remaining processes. This allows
# shutdown errors to be printed, while handling processes that fail to
# terminate quickly.
kill_subprocesses() {
  echo "Terminating subprocesses of PGID $1 excluding PID $$"
  # Get all PIDs in this process group except this process
  # (pgrep on NetBSD/MacOS does this by default, but Linux pgrep does not)
  # Uses a heredoc instead of piping to avoid including the grep PID
  pids=$(grep -Ev "\\<$$\\>" <<<"$(pgrep -g "$1")")
  if [ -n "$pids" ]; then
    echo "Terminating processes: ${pids//$'\n'/, }"
    # shellcheck disable=SC2086
    kill $pids || true
  fi
  sleep 2
  # Check for remaining processes and kill them
  pids=$(grep -Ev "\\<$$\\>" <<<"$(pgrep -g "$1")")
  if [ -n "$pids" ]; then
    echo "Killing remaining processes: ${pids//$'\n'/, }"
    # shellcheck disable=SC2086
    kill -9 $pids || true
  fi
}

# Terminate subprocesses on exit or interrupt
# shellcheck disable=SC2064
trap "kill_subprocesses $$" EXIT SIGINT SIGTERM

Answer 4

Universal solution which works also in sh (jobs there does not output anything to stdout):

trap "pkill -P $$" EXIT INT

Answer 5

I finally have found a solution that appears to work in all cases to kill all descents recursively regardless of whether they are jobs, or sub-processes. The other solutions here all seemed to fail with things such as:

while ! ffmpeg ....
do
  sleep 1
done

In my situation, ffmpeg would keep running after the parent script exited.

I found a solution here to recursively getting the PIDs of all child processes recursively and used that in the trap handler thus:

cleanup() {
    # kill all processes whose parent is this process
    kill $(pidtree $$ | tac)
}

pidtree() (
    [ -n "$ZSH_VERSION"  ] && setopt shwordsplit
    declare -A CHILDS
    while read P PP;do
        CHILDS[$PP]+=" $P"
    done < <(ps -e -o pid= -o ppid=)
    walk() {
        echo $1
        for i in ${CHILDS[$1]};do
            walk $i
        done
    }

    for i in "$@";do
        walk $i
    done
)

trap cleanup EXIT

The above placed at the start of a bash script succeeds in killing all child processes. Note that pidtree is called with $$ which is the PID of the bash script that is exiting and the list of PIDs (one per line) is reversed using tac to try and ensure that prarent processes are killed only after their children to avoid possible race conditions in loops such as the example I gave.

Answer 6

The trap 'kill 0' SIGINT SIGTERM EXIT solution described in @tokland's answer is really nice, but latest Bash crashes with a segmentation fault when using it. That's because Bash, starting from v. 4.3, allows trap recursion, which becomes infinite in this case:

1. shell process receives SIGINT or SIGTERM or EXIT;
2. the signal gets trapped, executing kill 0, which sends SIGTERM to all processes in the group, including the shell itself;
3. go to 1 :)

This can be worked around by manually de-registering the trap:

trap 'trap - SIGTERM && kill 0' SIGINT SIGTERM EXIT

The more fancy way that allows printing the received signal and avoids "Terminated:" messages:

#!/usr/bin/env bash

trap_with_arg() { # from https://stackoverflow.com/a/2183063/804678
  local func="$1"; shift
  for sig in "$@"; do
    trap "$func $sig" "$sig"
  done
}

stop() {
  trap - SIGINT EXIT
  printf '\n%s\n' "received $1, killing child processes"
  kill -s SIGINT 0
}

trap_with_arg 'stop' EXIT SIGINT SIGTERM SIGHUP

{ i=0; while (( ++i )); do sleep 0.5 && echo "a: $i"; done } &
{ i=0; while (( ++i )); do sleep 0.6 && echo "b: $i"; done } &

while true; do read; done

UPD: added a minimal example; improved stop function to avoid de-trapping unnecessary signals and to hide "Terminated:" messages from the output. Thanks Trevor Boyd Smith for the suggestions!

Answer 7

Update: https://stackoverflow.com/a/53714583/302079 improves this by adding exit status and a cleanup function.

trap "exit" INT TERM
trap "kill 0" EXIT

Why convert INT and TERM to exit? Because both should trigger the kill 0 without entering an infinite loop.

Why trigger kill 0 on EXIT? Because normal script exits should trigger kill 0, too.

Why kill 0? Because nested subshells need to be killed as well. This will take down the whole process tree.

Answer 8

Just for diversity I will post variation of https://stackoverflow.com/a/2173421/102484 , because that solution leads to message "Terminated" in my environment:

trap 'test -z "$intrap" && export intrap=1 && kill -- -$$' SIGINT SIGTERM EXIT

Answer 9

function cleanup_func {
    sleep 0.5
    echo cleanup
}

trap "exit \$exit_code" INT TERM
trap "exit_code=\$?; cleanup_func; kill 0" EXIT

# exit 1
# exit 0

Like https://stackoverflow.com/a/22644006/10082476, but with added exit-code

Answer 10

A nice version that works under Linux, BSD and MacOS X. First tries to send SIGTERM, and if it doesn't succeed, kills the process after 10 seconds.

KillJobs() {
    for job in $(jobs -p); do
            kill -s SIGTERM $job > /dev/null 2>&1 || (sleep 10 && kill -9 $job > /dev/null 2>&1 &)

    done
}

TrapQuit() {
    # Whatever you need to clean here
    KillJobs
}

trap TrapQuit EXIT

Please note that jobs does not include grand children processes.

Answer 11

I made an adaption of @tokland's answer combined with the knowledge from http://veithen.github.io/2014/11/16/sigterm-propagation.html when I noticed that trap doesn't trigger if I'm running a foreground process (not backgrounded with &):

#!/bin/bash

# killable-shell.sh: Kills itself and all children (the whole process group) when killed.
# Adapted from http://stackoverflow.com/a/2173421 and http://veithen.github.io/2014/11/16/sigterm-propagation.html
# Note: Does not work (and cannot work) when the shell itself is killed with SIGKILL, for then the trap is not triggered.
trap "trap - SIGTERM && echo 'Caught SIGTERM, sending SIGTERM to process group' && kill -- -$$" SIGINT SIGTERM EXIT

echo $@
"$@" &
PID=$!
wait $PID
trap - SIGINT SIGTERM EXIT
wait $PID

Example of it working:

$ bash killable-shell.sh sleep 100
sleep 100
^Z
[1]  + 31568 suspended  bash killable-shell.sh sleep 100

$ ps aux | grep "sleep"
niklas   31568  0.0  0.0  19640  1440 pts/18   T    01:30   0:00 bash killable-shell.sh sleep 100
niklas   31569  0.0  0.0  14404   616 pts/18   T    01:30   0:00 sleep 100
niklas   31605  0.0  0.0  18956   936 pts/18   S+   01:30   0:00 grep --color=auto sleep

$ bg
[1]  + 31568 continued  bash killable-shell.sh sleep 100

$ kill 31568
Caught SIGTERM, sending SIGTERM to process group
[1]  + 31568 terminated  bash killable-shell.sh sleep 100

$ ps aux | grep "sleep"
niklas   31717  0.0  0.0  18956   936 pts/18   S+   01:31   0:00 grep --color=auto sleep

Answer 12

To be on the safe side I find it better to define a cleanup function and call it from trap:

cleanup() {
        local pids=$(jobs -pr)
        [ -n "$pids" ] && kill $pids
}
trap "cleanup" INT QUIT TERM EXIT [...]

or avoiding the function altogether:

trap '[ -n "$(jobs -pr)" ] && kill $(jobs -pr)' INT QUIT TERM EXIT [...]

Why? Because by simply using trap 'kill $(jobs -pr)' [...] one assumes that there will be background jobs running when the trap condition is signalled. When there are no jobs one will see the following (or similar) message:

kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

because jobs -pr is empty - I ended in that 'trap' (pun intended).

Answer 13

jobs -p does not work in all shells if called in a sub-shell, possibly unless its output is redirected into a file but not a pipe. (I assume it was originally intended for interactive use only.)

What about the following:

trap 'while kill %% 2>/dev/null; do jobs > /dev/null; done' INT TERM EXIT [...]

The call to "jobs" is needed with Debian's dash shell, which fails to update the current job ("%%") if it is missing.

Answer 14

trap 'kill $(jobs -p)' EXIT

I would make only minor changes to Johannes' answer and use jobs -pr to limit the kill to running processes and add a few more signals to the list:

trap 'kill $(jobs -pr)' SIGINT SIGTERM EXIT

Answer 15

To clean up some mess, trap can be used. It can provide a list of stuff executed when a specific signal arrives:

trap "echo hello" SIGINT

but can also be used to execute something if the shell exits:

trap "killall background" EXIT

It's a builtin, so help trap will give you information (works with bash). If you only want to kill background jobs, you can do

trap 'kill $(jobs -p)' EXIT

Watch out to use single ', to prevent the shell from substituting the $() immediately.

Answer 16

So script the loading of the script. Run a killall (or whatever is available on your OS) command that executes as soon as the script is finished.