CODEWITHSUNDEEP
apache-kafkakafka-consumer-api
Jobin Ar
Jobin Ar

Reputation: 121

Is it possible to kill a consumer from the Kafka server?

When I check for consumer lag, it shows that a particular consumer-id is running from a particular host consuming from a topic. But when I go to that host there is no such consumer running.

How do I kill this consumer-id, so that I can reset consumer offset for the group that its part of.

Kafka server version: 0.11.0.1

Kafka client version(stuck): 0.10.0.2

This consumer-id got stuck in the first place as it was not able to consume messages because of some messages having headers in Kafka.

I've already tried the following:

  1. Consuming from a different host and different Kafka version, it consumes messages but the consumer-id, host does not change.
  2. Restarting kafka broker which is the leader for that topic.
  3. Changing security groups to prevent the host from connecting to my broker.

Upvotes: 3

Views: 4634

Answers (1)

tgrez
tgrez

Reputation: 714

Perhaps what you see is not a consumer id, but a consumer group, see Kafka docs, consumer config to learn about the difference.

Kafka uses consumer groups to keep track of the last consumed message (consumer offset), so when talking about the consumer lag this is probably the explanation.

This means there is no consumer running and you only need to get rid of the consumer offset for this group. See e.g. How do I delete a Kafka Consumer Group to reset offsets?

Upvotes: 1

Related Questions