CODEWITHSUNDEEP
apache-kafka
Raman
Raman

Reputation: 717

Kafka: Offset sequence

Will each message posted (assuming messages are compressed) have a unique offset# assigned and remain sequential and incremental within a given partition? Also, is it possible that the offset# sequence can reset itself, meaning it can start from 0?

Upvotes: 2

Views: 3419

Answers (1)

Giorgos Myrianthous
Giorgos Myrianthous

Reputation: 39950

Official documentation is quite clear on that:

For each topic, the Kafka cluster maintains a partitioned log that looks like this:

enter image description here

Each partition is an ordered, immutable sequence of records that is continually appended to—a structured commit log. The records in the partitions are each assigned a sequential id number called the offset that uniquely identifies each record within the partition.

Consumers are able to read from specific offsets even from the beginning:

enter image description here

In order to reset offsets for a particular consumer group you can use the following: 

kafka-consumer-groups --bootstrap-server localhost:9092 --group my-group --reset-offsets --to-earliest --all-topics --execute

Upvotes: 4

Related Questions