CODEWITHSUNDEEP
node.jsapache-kafka
санжар кулиев
санжар кулиев

Reputation: 59

I need a way for Kafka consumer to commit a particular message

Let's imagine a stream of messages in a kafka partition.

1, 2, 3, 4, 5

Is there a way for a consumer to commit one message(let's say 2), so that if that particular consumer started reading messages from the beginning all over again, it would only consume uncommitted messages?

1, 3, 4, 5

Upvotes: 1

Views: 1536

Answers (2)

OneCricketeer
OneCricketeer

Reputation: 191820

I'm not sure I understand the use case, but you could use a compacted topic

Say you send these messages 

(1, 1)
(2, 2)
(3, 3) 
(4, 4)
(5, 5)

And you don't want to see value 2 anymore, then you send one more record 

(2, null)

After compaction, all keys with 2 get removed

Upvotes: 1

codejitsu
codejitsu

Reputation: 3182

No. Apache Kafka doesn't support single record acknowledgement AFAIK. So, when a message gets acknowleged (say 3), all prevoiusly seen messages are considered read.

You can process your topic with some filter processor which will drop messages by some condition. The output topic of this processor could be consumed by your app then.

Upvotes: 1

Related Questions