CODEWITHSUNDEEP
apache-kafka
Hayi
Hayi

Reputation: 6236

Offsets retention with running consumers

in Kafka doc offsets.retention.minutes is defined as

After a consumer group loses all its consumers (i.e. becomes empty) its offsets will be kept for this retention period before getting discarded. For standalone consumers (using manual assignment), offsets will be expired after the time of last commit plus this retention period.

If a running consumer don't commit its offsets for 2 days with offsets.retention.minutes set to 1 day

Would the Consumer group offsets be discarded then ? so if the consumer restarts the offsets will depend on auto.offset.reset ?

or the offsets.retention.minutes config is applied only when consumers are down ?

Upvotes: 0

Views: 204

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191725

The offsets topic works like any other - it's keyed by group id and compacted based on this.

If there are no new events into the topic, then group information and offsets get discarded

Consumer heartbeats don't create new offset records, only offset commits do

Upvotes: 1

Related Questions