CODEWITHSUNDEEP
linuxbashprocessparallel-processinggnu-parallel
Yibo Yang
Yibo Yang

Reputation: 2413

How do I terminate GNU parallel without killing running jobs?

I'm running a bunch of shell scripts like parallel -a my_scripts bash and at some point I decided I've run enough of them and would like to stop spawning new jobs, and simply let all the existing jobs finish. Put another way, I want to kill the parent process without killing the children.

There seems to be ways of controlling termination when first launching GNU parallel (for example if I know in advance I only want to run x jobs, then I can use --halt now,success=x argument), but I couldn't find how to control GNU parallel when it is already running.

Sure I can just CTRL+C to kill parallel, and rerun the jobs that were aborted, but I thought there might be a smarter way.

Upvotes: 10

Views: 5600

Answers (2)

Yibo Yang
Yibo Yang

Reputation: 2413

Update:

If you have a new version of parallel >= 20190322: as per @Bowi's comment, "since 2019, it is no longer SIGTERM, it's SIGHUP instead:

kill -HUP $PARALLEL_PID

SIGTERM now terminates the children without letting them finish."

See:

Original answer:

I figured it out. The answer is to simply send SIGTERM to the parent parallel process (just killing its PID will do). parallel then responds with the following (in this case I have 4 jobs running):

parallel: SIGTERM received. No new jobs will be started.
parallel: Waiting for these 4 jobs to finish. Send SIGTERM again to stop now.

I dug it out of the man page:

COMPLETE RUNNING JOBS BUT DO NOT START NEW JOBS
If you regret starting a lot of jobs you can simply break GNU parallel, but if you want to make sure you do not have half-completed jobs you should send the signal SIGTERM to GNU parallel:

killall -TERM parallel

This will tell GNU parallel to not start any new jobs, but wait until the currently running jobs are finished before exiting.

Upvotes: 16

Martin Vít Vavřík
Martin Vít Vavřík

Reputation: 23

Small addition if you use parallel inside a simple script:
If you use parallel (<= 20190322) in a simple script.sh that looks something like this:

#!/bin/bash
cd $(dirname $0)

seq -w 0 $(($1-1)) | parallel -j+0 --progress ./something.exe {}

you can look up the PID of the child process of script.sh (using pgrep -P $scriptsh_pid) and then use

kill -TERM $child_process_pid

If I only run script.sh in my shell, killall -TERM perl also works.

Upvotes: 0

Related Questions