CODEWITHSUNDEEP
apache-kafkaapache-kafka-streams
Ankur rana
Ankur rana

Reputation: 690

out-of-order handling of messages in Kafka streams

I recently came across this article on Apache Kafka documentation regarding Handling of out of order messages in Kafka streams

https://kafka.apache.org/21/documentation/streams/core-concepts#streams_out_of_ordering

Can someone explain me the reason behind the below statement :

Within a topic-partition, a record's timestamp may not be monotonically increasing along with their offsets. Since Kafka Streams will always try to process records within a topic-partition to follow the offset order, it can cause records with larger timestamps (but smaller offsets) to be processed earlier than records with smaller timestamps (but larger offsets) in the same topic-partition.

As per my understanding, if messages in a single partition are produced by a single producer, their timestamp must be in nondecreasing order. How come messages are not in monotonically increasing order?

Upvotes: 3

Views: 6407

Answers (1)

Val Bonn
Val Bonn

Reputation: 1189

A Kafka Producer may specify a timestamp when it produces a message.

An example with the standard Java Producer API: https://kafka.apache.org/21/javadoc/org/apache/kafka/clients/producer/ProducerRecord.html#ProducerRecord-java.lang.String-java.lang.Integer-java.lang.Long-K-V-

Upvotes: 2

Related Questions