CODEWITHSUNDEEP
apache-kafkaapache-zookeeper
Judy
Judy

Reputation: 1871

2 cluster of zookeper servers in hadoop+kafka cluster - is it posible?

We have Kafka cluster with the following details

3 kafka machines

3 zookeeper servers

We also have Hadoop cluster that includes datanode machines And all application are using the zookeeper servers, including the kafka machines

Now

We want to do the following changes

We want to add additional 3 zookeeper servers that will be in a separate cluster

And only kafka machine will use this additional zookeeper servers

Is it possible ?

Upvotes: 0

Views: 185

Answers (2)

OneCricketeer
OneCricketeer

Reputation: 191743

Editing the ha.zookeeper.quorum in Hadoop configurations to be separate from zookeeper.connect in Kafka configurations, such that you have two individual Zookeeper clusters, can be achieved, yes.

However, I don't think Ambari or Cloudera Manager, for example, allow you to view or configure more than one Zookeeper cluster at a time.

Upvotes: 1

sev7e0
sev7e0

Reputation: 106

Yes, that's possible. Kafka uses Zookeeper to perform various distributed coordination tasks, such as deciding which Kafka broker is responsible for allocating partition leaders, and storing metadata on topics in the broker.

After closing kafka, the original zookeeper cluster data will be copied to the new cluster using tools, this is a zookeeper cluster data transfer util zkcopy

But if your Kafka cluster didn't stop work, you should think about Zookeeper data transfer to additional zookeeper servers.

Upvotes: 1

Related Questions