CODEWITHSUNDEEP
apache-kafkamessagingkafka-consumer-api
tpuli
tpuli

Reputation: 435

Apache Kafka: Can we restrict message to be read by only 1 consumer?

I am new to Kafka and the scenario that i am trying to achieve using Apache Kafka is:

I have a 3 producers who are sending instructions to a particular topic in Kafka. I have around 35 consumers who picks up one instruction at a time and performs the task mentioned in the instruction set. Once completed it will post the updates to an another topic in Kafka.

Requirements.

  1. An instruction read by 1 consumer should not be given to another for processing
  2. Message Priority * - Not a mandatory requirement
  3. Only 1 instruction should be picked up at a time for processing.

I know that this can be achieved using JMS. But my organisation already has a Kafka setup ready as part of Cloud-era HDFS setup. So the using Kafka to achieve this is most preferred.

I have googled around to find a solution for this. But i was not able to find a proper answer to my 1st and most important requirement.

Any suggestions? And thanks in advance.!

Upvotes: 8

Views: 6042

Answers (1)

Lundahl
Lundahl

Reputation: 6552

All you have to do is to put the consumers in the same consumer group by assigning the same group.id to them. Only one instance of a consumer in the same group will get a message.

Refer to documentation at https://kafka.apache.org/documentation#intro_consumers

Upvotes: 10

Related Questions