CODEWITHSUNDEEP
apache-kafka
Saurabh wagh
Saurabh wagh

Reputation: 61

Can we get the value from Kafka using key? How we can pass header in Kafka Message?

  1. I have just started the exploring the Kafka, I am not able to understand what is the need of key value in kafka,as we can also directly send a message as single string like JMS. And is it possible to get the value from records by specifying the key? (I am comparing this with selector in JMS, so based upon that we can filter the message).
  2. How we can pass header with message? And is it possible just send a message with header rather than key value pair?

Upvotes: 0

Views: 1323

Answers (2)

OneCricketeer
OneCricketeer

Reputation: 191844

The key is optional, and, by default, determines which partition the messages arrives at in the topic.

It is not possible to select on a key from a topic - you'd have to compute the partition of it, then scan all messages of a topic-partition. It is possible to create a Ktable in the Kafka Streams API to create a type of KV store that you can select from, though, via "Interactive Queries")

The Java API has a producer send method that allows you to pass headers. You would need to set the key and value to null to only send headers. I'm not familiar enough with other client support for headers

Headers can be used to send extra metadata, for example, client version numbers, or traces to be used in services like zipkin. This data lies outside of the business context of what you'd send as part of the key/value payload. And I'm not aware of being able to send headers on the console

Upvotes: 2

Jacek Laskowski
Jacek Laskowski

Reputation: 74709

what is the need of key value in kafka

A key determines the partition a message is produced to.

Every message has to have a key that could be specified directly (while creating a ProducerRecord using Kafka Producer API) or indirectly via a Partitioner that "computes the partition for the given record."

Kafka Producer API gives us DefaultPartitioner that uses a 32-bit murmur2 hash to compute the partition for a record (with the key defined) or chooses a partition in a round-robin fashion (per the available partitions of the topic).

as we can also directly send a message as single string like JMS.

That's possible with a custom Partitioner. By default, that's not the case.

Upvotes: 0

Related Questions