CODEWITHSUNDEEP
javaapache-kafka
user3198603
user3198603

Reputation: 5836

Consumer connecting to zookeeper not broker for message consumption?

Per this document at Kafka tutoria

Zookeper is started at port 2181 Broker/Kafka started at 9092

Start Producer to Send Messages

    bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Hello-Kafka // producer is producing directly to broker not zookeeper which is correct

Start Consumer to Receive Messages

    bin/kafka-console-consumer.sh --zookeeper localhost:2181 —topic Hello-Kafka --from-beginning

Per mine understanding consumer consumes message directly from broker but in above consumer command we did not mention broker but only zookeeper. Will consumer connect to zookeeper(instead of broker) to consume message ?

Upvotes: 3

Views: 6881

Answers (3)

Hans Jespersen
Hans Jespersen

Reputation: 8335

There is as old Kafka consumer (0.8.2 and earlier) and a new Kafka consumer (0.9 and above). For a great description of how the new consumer works see the original announcement blog here

https://www.confluent.io/blog/tutorial-getting-started-with-the-new-apache-kafka-0-9-consumer-client/

The old consumer would connect to zookeeper but still would fetch all messages from Kafka.

The new consumer has no zookeeper dependency and does not connect to zookeeper at all.

The console-producer and console-consumer supports both the old an new apis depending on which options you give it. The example console-consumer you have provided is the old consumer because it specifies --zookeeper instead of --bootstrap-server

Upvotes: 5

ppatierno
ppatierno

Reputation: 10075

What are you using with the --zookeeper parameter is the old consumer. The new Kafka consumer (you should use it) connects to a Kafka broker directly. For doing this, you can use the --bootstrap-server option instead of the --zookeeper specifying localhost:9092. The new consumer doesn't save offset on Zookeeper anymore but on a Kafka broker to a specific topic named __consumer_offsets.

Upvotes: 4

Rahul Kumar
Rahul Kumar

Reputation: 148

Consumer subscribe to topic and connect to zookeeper.The Kafka servers share information via a Zookeeper cluster. Kafka stores basic metadata in Zookeeper such as information about topics, brokers, consumer offsets (queue readers) and so on.For better understating for the role of zookeeper look for below link.

https://www.quora.com/What-is-the-actual-role-of-ZooKeeper-in-Kafka

Upvotes: -1

Related Questions