Andy
Andy

Reputation: 613

How to kill a stale process

I run transcoding on Ubuntu server and sometimes the process goes stale and provides no output but ps aux shows process is still running - but with stale CPU usage

I can get the CPU usage with the following command(ffmpeg process i was working with has process id 4416:

ps aux | grep -v grep | grep 4416 | awk '{print $3}'

and I could probably create a small script to kill a specific process but how would I create a loop that would check each ffmpeg process and kill it if its stale(runit will restart it afterwards)?

I think it would need to execute the command to get CPU usage twice with a minute cron and kill the process if CPU usage is the same. Would would I do this?

Upvotes: 2

Views: 6425

Answers (1)

ShellFish
ShellFish

Reputation: 4551

Ok so we do a couple ps aux requests and then we'll check for stale processes.

#!/bin/bash

# setup, change this if undesired
workdir="/tmp/.stale_process_killing"
[ -d "$workdir" ] || mkdir "$workdir"

# get ps aux outputs, change number of iterations if desired
for ((i=1; i<5; i++)); do
    # add file number to file
    echo $i > "${workdir}/psaux${i}"
    # add ps output to file
    ps aux | grep 'ffmpeg' >> "${workdir}/psaux${i}"
    # change this timeout to suit your needs
    sleep 1
done

# now parse the files using awk
awk '
    FNR==1 { ix = $1 }
    FNR!=1 { cpu[$2][ix] = $3 }
    END {
        for (pid in cpu) { 
            j=1;
            while (cpu[pid][j] == cpu[pid][j+1] && j <= ix) {
                if (cpu[pid][j++] == "") {
                    j=1;
                    break;
                }
            }
            if (j >= ix) {
                system("kill " pid);
            }
        }
    }' "${workdir}/psaux"*

Version containing debug prints, to safely check.

#!/bin/bash

# setup, change this if undesired
workdir="/tmp/.stale_process_killing"
[ -d "$workdir" ] || mkdir "$workdir"

# get ps aux outputs, change number of iterations if desired
for ((i=1; i<5; i++)); do
    # add file number to file
    echo $i > "${workdir}/psaux${i}"
    # add ps output to file
    ps aux | grep 'ffmpeg' >> "${workdir}/psaux${i}"
    # change this timeout to suit your needs
    sleep 1
done

# now parse the files using awk
awk '
    FNR==1 { ix = $1 }
    FNR!=1 { cpu[$2][ix] = $3 }
    END {
        for (pid in cpu) { 
            j=1;
            while (cpu[pid][j] == cpu[pid][j+1] && j <= ix) {
                print cpu[pid][j]
                if (cpu[pid][j++] == "") {
                    j=1;
                    break;
                }
            }
            if (j >= ix) {
                print "kill " pid;
            } else {
                print "no kill " pid;
            }
        }
    }' "${workdir}/psaux"*

Upvotes: 2

Related Questions