CODEWITHSUNDEEP
apache-kafka
Joe
Joe

Reputation: 47709

Is there anything special about Kafka's message keys?

I see no mention of message keys (org.apache.kafka.clients.producer.ProducerRecord.key) except that they may be used for topic partitioning.

Am I free to put whatever data I like in the key or are there some special semantics that I should conform to?

The key seems like a convenient header to put small amounts of metadata. Is this a bad idea?

Upvotes: 6

Views: 4131

Answers (1)

Sudhesh Rajan
Sudhesh Rajan

Reputation: 367

I see no mention of message keys (org.apache.kafka.clients.producer.ProducerRecord.key) except that they may be used for topic partitioning.

Use of key in Kafka: 1. Partitioning 2. guarantees ordering within a partition 3. Used during log compaction to create offset map

may be more which I haven't learnt yet.. :-)

Am I free to put whatever data I like in the key or are there some special semantics that I should conform to?

Key can be of any type Null, string or any other form with valid serialization mechanism.

do you know if the size of the key matters (e.g. 64 bytes vs 200 bytes)

Size of the key will matter as it will change your overall payload size. Therefore each message utilization on the buffer will change. I can't think of any other impact.

Upvotes: 4

Related Questions