CODEWITHSUNDEEP
kubernetes
David Parks
David Parks

Reputation: 32081

How to get 1 initContainer when parallelism is set > 1

I have a job in Kubernetes as defined below (with some omissions for brevity). Parallelism and N completions are set. I want 1 init container to delay the start of the parallel containers. When I run it as-is I get an init container per each completion it appears.

apiVersion: batch/v1
kind: Job
metadata:
  name: my-job
spec:
  parallelism: 10
  completions: 10
  backoffLimit: 0
  template:
    spec:

      # I want 1 of these to run first
      initContainers:
      - name: init-container
        image: my/container:latest
        imagePullPolicy: Always
        command: ["init_script.sh"]

      # I want 10 of these to run in parallel once init_script.sh exits
      containers:
      - name: container
        image: my/container:latest
        imagePullPolicy: Always
        command: ["run_job.sh"]

      restartPolicy: Never

Upvotes: 1

Views: 940

Answers (1)

Rico
Rico

Reputation: 61551

Kubernetes Jobs are very basic and don't provide any advanced scheduling mechanism with scheduling dependencies. For example, the ability to start a job after another job has finished. My advice is to use a more advanced scheduling tool on top of Kubernetes. There are few in open source that you can use. For example.

Two other alternatives:

  • You can even create your own custom scheduler.
  • You can build your own operator that kicks off your regular jobs based on status.

✌️

Upvotes: 3

Related Questions