CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
f1sherox
f1sherox

Reputation: 349

Kafka consumer rebalancing condition

What if I create a consumer C1 with group consumerGroup to read data from topic A. And some time later create consumer C2, in same group, to read data from topic B.

Will creation of consumer C2 trigger rebalance? As a more general question, when kafka will perform rebalance?

Upvotes: 4

Views: 2485

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62330

Each time a new consumer joins the group or a consumer leaved the group (actively by calling close() or via timeout) a rebalance will be triggered.

Furthermore, if you subscribe to a topic that is not created yet, a rebalance will be triggered after the topic got created. Same, if a topic you subscribed to gets deleted. Also if the number of partitions for any subscribed topics get changed. Last but not least, if you subscribe via pattern, if a new topic matches the pattern or a matching topic gets deleted or if the number of partitions get changed for any matching topic, rebalance will happen.

See https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.9+Consumer+Rewrite+Design

Upvotes: 7

Related Questions