CODEWITHSUNDEEP
apache-kafka
Knows Not Much
Knows Not Much

Reputation: 31526

Getting the list of Brokers Dynamically

I am new to Kafka. I wrote my first message producer like this

private Properties kafkaProps = new Properties();
kafkaProps.put("bootstrap.servers", "broker1:9092,broker2:9092");
kafkaProps.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
kafkaProps.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
producer = new KafkaProducer<String, String>(kafkaProps);

is it possible that I bootstrap to Kafka and ask it what brokers are present and then put them in the properties? Otherwise there will always be a problem that the config of the code is out of sync with the actual state of the cluster.

Upvotes: 1

Views: 689

Answers (1)

theDima
theDima

Reputation: 756

Normally, kafka brokers will be managed by a zookeeper, so you can access the zookeeper instance to gather information about kafka brokers.

You can use, for example, a org.apache.storm.shade.org.apache.zookeeper.ZooKeeper instance to connect to Zookeeper, and run its getChildren(pathToBrokerIds, false) method which returns a list of kafka brokers' ids. Then you can run zookeeper's getData(..) method with each id as an argument, and get the info for that broker, including host and port.

Upvotes: 1

Related Questions