CODEWITHSUNDEEP
apache-kafka
Shengxin Huang
Shengxin Huang

Reputation: 707

Can Kafka message consumed by different consumers?

I am new to Kafka. I have produced a kafka message named foo by kafkaSink ,a class in Flume.When I want to consume the messages,many questions come unclear to me: 1.I have tried to use kafka-console-consumer to consume message foo,and it succeeded.Can I consume message foo again with another consumer process somewhere else? 2.To the opposite,I don't want to consume message foo again, so when I try to consume with another consumer,I should obtain nothing.How can I achieve this? 3.What if there are two messages foo and bar.Can I specify consumers precisely?(For example,I want process A consumes message foo and process B consumes message bar.To go further ,can O specify a range of message offsets? Does it have something to do with consumer group?

Upvotes: 1

Views: 1138

Answers (1)

Balu Vyamajala
Balu Vyamajala

Reputation: 10333

  1. Can I consume message foo again with another consumer process somewhere else?

Yes. It can be consumed as many times as we want, either by using a new consumer group or by resetting the offset of the existing consumer group.

2.To the opposite,I don't want to consume message foo again, so when I try to consume with another consumer,I should obtain nothing.How can I achieve this?

It's all tied to a consumer group name, which typically tied to one application that is needing these messages. we need to keep the same consumer group name, and typically commited offset is retained for a week(can be changed), so, we can run the app n no of times from n different places by keeping same consumer group name, we will not consume it again, unless we reset the offset.

3.What if there are two messages foo and bar.Can I specify consumers precisely?(For example,I want process A consumes message foo and process B consumes message bar.To go further ,can O specify a range of message offsets? Does it have something to do with consumer group?

We can always consume particular message of a given partition and offset by positioning the consumer group at that offset. Its called seeking an offset, rather than seeking to earliest or latest.

Upvotes: 3

Related Questions