CODEWITHSUNDEEP
kubernetesapache-kafka
Pooja
Pooja

Reputation: 501

What is the recommended way to deploy kafka to make it deploy in all the available nodes?

I have 3 nodes in k8s and i'm running kafka (3 cluster). While deploying zk/broker/rest-proxy, its not getting deployed in all the available nodes. How can i make sure that all pods are deployed in different nodes. Do i need to use nodeaffinity or podaffinity ?

Upvotes: 1

Views: 450

Answers (2)

Max Lobur
Max Lobur

Reputation: 6040

I recommend using soft anti-affinity which will look like: 

affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values:
              - <your app label>
          topologyKey: kubernetes.io/hostname
        weight: 100

Here I explained the difference between anti-affinity types with examples applied to a live cluster: https://blog.verygoodsecurity.com/posts/kubernetes-multi-az-deployments-using-pod-anti-affinity/

Upvotes: 1

Vasilii Angapov
Vasilii Angapov

Reputation: 9022

If you want all pods to run on different nodes - you must use PodAntiAffinity. If this is hard requirement - you must use requiredDuringSchedulingIgnoredDuringExecution rule. If it's not - use preferredDuringSchedulingIgnoredDuringExecution.

topologyKey should be kubernetes.io/hostname.

In labelSelector put your pod's labels.

Upvotes: 2

Related Questions