CODEWITHSUNDEEP
kubernetesgoogle-kubernetes-engine
Sunil Kumar
Sunil Kumar

Reputation: 1711

Allow only one pod of a type on a node in Kubernetes

How to allow only one pod of a type on a node in Kubernetes. Daemon-sets doesn't fit into this use-case.

For e.g. - Restricting scheduling of only one Elasticsearch pod on a node, to prevent data loss in case the node goes down.

It can be achieved by carefully planning CPU/memory resource of pod and machine type of cluster.

Is there any other way to do so?

Upvotes: 15

Views: 26301

Answers (4)

holger
holger

Reputation: 11

Is creating one elasticsearch deploy per node an option? In case yes I found it the easiest to use the nodeSelector combined with the Always restart policy. You can match on different labels, here I simply use the zone of the Azure AvailabilitySet. E.g. like this

spec: containers: ... nodeSelector: failure-domain.beta.kubernetes.io/zone: "2" ... restartPolicy: Always

Upvotes: 0

Bastien Libersa
Bastien Libersa

Reputation: 690

Kubernetes 1.4 introduced Inter-pod affinity and anti-affinity. From the documentation: Inter-pod affinity and anti-affinity allow you to constrain which nodes your pod is eligible to schedule on based on labels on pods that are already running on the node.

That won't prevent a pod to be scheduled on a node, but at least the pod will be scheduled on the node if and only if the scheduler has no choice.

Upvotes: 6

Severin Pappadeux
Severin Pappadeux

Reputation: 20080

I'm running MC jobs on kubernetes/GCE cluster, and for me M:N scheduling is important in a sense that out of M jobs, I want one job/pod per node for N nodes running (M >> N).

For me solution was to have explicit CPU limit set in pod JSON file

"resources": {
    "limits": {
        "cpu": "700m"
    }
}

ANd I have no replication controller, just pure batch-style cluster. Numbe of nodes N is typically 100-200-300, M is about 10K-20K

Upvotes: 1

Robert Bailey
Robert Bailey

Reputation: 18200

If you assign a constraint to your pod that can only be met at most once per node, then the scheduler will only be able to place one pod per node. A good example of such a constraint is a host port (the scheduler won't try to put two pods that both require the same host port onto the same node because the second one will never be able to run).

Also see How to require one pod per minion/kublet when configuring a replication controller?

Upvotes: 2

Related Questions