CODEWITHSUNDEEP
pythonapache-kafkakafka-consumer-apikafka-python
hasherBaba
hasherBaba

Reputation: 339

How to read messages after the last consumed message in case of consumer shutdown in Kafka?

I have a way of performing this task in shell: How to make kafka consumer to read from last consumed offset but not from beginning

However, I am willing to do this in Python, using kafka-python

I could not find any api for this case.

http://kafka-python.readthedocs.io/en/latest/apidoc/KafkaConsumer.html

Upvotes: 2

Views: 929

Answers (1)

Mickael Maison
Mickael Maison

Reputation: 26885

To enable Consumer Groups with kafka-python you just need to set the group_id configuration in the Consumer.

From the Consumer API:

group_id (str or None) – The name of the consumer group to join for dynamic partition assignment (if enabled), and to use for fetching and committing offsets. If None, auto-partition assignment (via group coordinator) and offset commits are disabled. Default: None

If you set that to any value, the Consumer will automatically commit the offsets it reads and restart from that position in case it's shutdown.

Upvotes: 2

Related Questions