CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
Himalay Majumdar
Himalay Majumdar

Reputation: 3973

Kafka Consumer v.10

There were some disadvantages of using The SimpleConsumer in the previous versions of Kafka 0.8

  1. You must keep track of the offsets in your application to know where you left off consuming.
  2. You must figure out which Broker is the lead Broker for a topic and partition
  3. You must handle Broker leader changes

In the latest version of Kafka we have a unified Consumer which polls to get messages. How should one track and handle broker/leader changes?

Upvotes: 1

Views: 265

Answers (1)

Manav Garg
Manav Garg

Reputation: 522

In the new version, for consumer offset management, we have a default topic called "__consumer_offsets", rather than relying on zookeeper or manually managing it as in the 0.8 version.

The broker leader changes is still handled through zookeeper. Remember, even in the current version, Kafka does need zookeeper to work. It is just the consumer offset dependency which has changed. So, zookeeper tracks the broker/leader election.

Also, as a consumer, you really need not worry about broker leader changes. If you are using a client with high level API for consumer, you only need to specify Consumer Group ID, Topic to consume and broker list (few servers to identify the cluster, the topic might not even reside there)

Upvotes: 1

Related Questions