CODEWITHSUNDEEP
apache-kafka
xRobot
xRobot

Reputation: 26567

Why does the cleanup.policy of the topic __consumer_offsets is COMPACT?

Why that topic is COMPACT and not, for example, DELETE+COMPACT?

Is there a specific reason?

Upvotes: 0

Views: 816

Answers (1)

nipuna
nipuna

Reputation: 4105

Because, If you run a group consumer, and it stops for some time, After restating group consumer, it should consume from last offset it is consumed.

__consumer_offsets topics keeps your group consumer's last offset for each partition it is consumed. So if it deleted from the kafka, there is no trace for that.

cleanup.policy = compact means it will keep the latest message for a key any long untill you tomstone it.

So, like this, your grouo consumer's last offset is secured in __consumer.offsets because it is compacted.

Upvotes: 2

Related Questions