CODEWITHSUNDEEP
kubernetescontainersjobskubernetes-jobs
Bosh
Bosh

Reputation: 8738

How to set a time limit for a Kubernetes job?

I'd like to launch a Kubernetes job and give it a fixed deadline to finish. If the pod is still running when the deadline comes, I'd like the job to automatically be killed.

Does something like this exist? (At first I thought that the Job spec's activeDeadlineSeconds covered this use case, but now I see that activeDeadlineSeconds only places a limit on when a job is re-tried; it doesn't actively kill a slow/runaway job.)

Upvotes: 23

Views: 32098

Answers (3)

th0masb
th0masb

Reputation: 309

You could instead add the activeDeadlineSeconds to the pod spec in the pod template defined as part of the job. This way the pods which are spawned by the job are limited with the timeout.

Upvotes: 1

tmetodie
tmetodie

Reputation: 648

From the way I understand the documentation of activeDeadlineSeconds section is that it refers to the active time of a Job and after this time the Job is considered Failed.

Official doc statement:

The activeDeadlineSeconds applies to the duration of the job, no matter how many Pods are created. Once a Job reaches activeDeadlineSeconds, all of its running Pods are terminated and the Job status will become type: Failed with reason: DeadlineExceeded

https://kubernetes.io/docs/concepts/workloads/controllers/job/#job-termination-and-cleanup

Upvotes: 19

ahmet alp balkan
ahmet alp balkan

Reputation: 45216

You can self-impose timeouts on the container's entrypoint command by using GNU timeout utility.

For example the following Job that computes first 4000 digits of pi will time out after 10 seconds:

apiVersion: batch/v1
kind: Job
metadata:
  name: pi
spec:
  template:
    metadata:
      name: pi
    spec:
      containers:
      - name: pi
        image: perl
        command: ["/usr/bin/timeout", "10", "perl", "-Mbignum=bpi", "-wle", "print bpi(4000)"]
      restartPolicy: Never

(Manifest adopted from https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#running-an-example-job)

You can play with the numbers and see it timeout or not. Typically computing 4000 digits of pi takes ~23 seconds on my workstation, so if you set it to 5 seconds it'll probably always fail and if you set it to 120 seconds it will always work.

Upvotes: 17

Related Questions