CODEWITHSUNDEEP
apache-kafkaapache-zookeeper
Judy
Judy

Reputation: 1871

how many zookeper servers we need in order to support 18 kafka machines

6 kafka machines ( they are physical machines - DELL HW )

3 zookeeper server

we want to add 12 kafka machines to the cluster

in that case how many zookeeper server should be ?

in order to support 18 kafka machines ?

Upvotes: 0

Views: 190

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

Well, your question was tagged with Hadoop, but for Kafka alone, 3 will "work", but 5-7 is "better".

But, these should be dedicated Zookeeper servers for Kafka, and not shared with Hadoop services such as the namenode, Hive, HBase, etc. Especially on the level of 30+ Hadoop servers. This is because Zookeeper is very latency specific, and needs lots of memory to handle these types of processes.

This can easily be done in Ambari with specific server configs, but not letting Ambari use its templates to populate the single Zookeeper quorum that it tracks (which is somewhat painful to find in every service, that it's really worth not using Ambari at all for configs, and rather Puppet or Ansible, etc, but I digress)

Keep in mind, your cluster will be 1/3 entirely unbalanced, and adding brokers will not move existing data or cause replicas to get assigned to the new brokers for existing topics

Upvotes: 1

Related Questions