CODEWITHSUNDEEP
kubernetesjob-schedulingstream-processing
Horttanainen
Horttanainen

Reputation: 93

How to properly use Kubernetes for job scheduling?

I have the following system in mind: A master program that polls a list of tasks to see if they should be launched (based on some trigger information). The tasks themselves are container images in some repository. Tasks are executed as jobs on a Kubernetes cluster to ensure that they are run to completion. The master program is a container executing in a pod that is kept running indefinitely by a replication controller.

However, I have not stumbled upon this pattern of launching jobs from a pod. Every tutorial seems to be assuming that I just call kubectl from outside the cluster. Of course I could do this but then I would have to ensure the master program's availability and reliability through some other system. So am I missing something? Launching one-off jobs from inside an indefinitely running pod seems to me as a perfectly valid use case for Kubernetes.

Upvotes: 1

Views: 2604

Answers (1)

frankgreco
frankgreco

Reputation: 1516

Your master program can utilize the Kubernetes client libraries to preform operations on a cluster. Find a complete example here.

Upvotes: 2

Related Questions