CODEWITHSUNDEEP
apache-kafka-streamsktable
Utkarsh Sharma
Utkarsh Sharma

Reputation: 51

Retention period with global K tables in kafka streams

Does a Kafka streams app with global K tables only retain records as per what is defined in the input topic retention period?

I know in a normal K-table with changelog enabled, it creates a changelog topic with an infinite retention period. I wanted to understand how it works with a global K table...

Upvotes: 0

Views: 25

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62285

Neither KTable nor GlobalKTable use any retention time (expect windowed-KTables), but KTable changelog topics are configured with log-compaction instead (not that Kafka Streams creates and configures the changelog topics).

For GlobalKTables, the same apply: you should consume from a compacted topic (for this case, the topic is an input topic, ie, "user topic" and it's your responsibility to configure it correctly). If the global input topic is configured with retention time it would not be known by the GlobalKTable. A GlobalKTable only deletes rows if it reads an input delete/tombstone record (<key, null>).

Upvotes: 0

Related Questions