CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
Harald
Harald

Reputation: 5113

Read keys only from Kafka

Is it possible to read only the keys from Kafka? We have an application where the values stored in the Kafka log are quite big. In order to debug and quickly check whether a certain message is (still) in the log and at which offset, it would be great to just fetch and grep through the keys instead of reading the whole message value. Just discarding the value on the consumer side would be a big waste of time and bandwidth?

Can we get the keys only? How? Java solutions preferred, but Scala would be fine too.

Upvotes: 5

Views: 2587

Answers (1)

serejja
serejja

Reputation: 23881

As per Kafka Wire Protocol there is no possibility to fetch keys or values only. The fetch request does not contain any information to request only for keys or values, thus the returned message set will contain all keys and values present for returned messages.

You could sure filter out keys/values on the client side, but currently, I don't see any possibility to avoid network overhead you are looking for.

Upvotes: 5

Related Questions