CODEWITHSUNDEEP
error-handlingapache-kafkaconsumerpausekafkajs
Kha Nguyễn
Kha Nguyễn

Reputation: 81

How to pause KafkaJS consumer and retry after sometime

I'm using KafkaJS ( https://kafka.js.org ) to connect to a Kafka cluster. My consumers are responsible for processing tasks which can fail at some time and succeed after waiting for a duration (for example, 1 hour). Before this duration, any attempt to process any task will result to another failure.

I tried to use Pause & Resume method as described at https://kafka.js.org/docs/consuming#a-name-pause-resume-a-pause-resume. However, the consumer restart immediately and keep consuming the failed task again.

How can I pause consuming messages without committing a failed one and restarting the consumer immediately?

Upvotes: 4

Views: 1916

Answers (1)

ashish
ashish

Reputation: 318

An alternative approach could be to save the last fetched offset at some cache or DB and then call the seek method on consumer to start fetching from the last offset.

consumer.seek()

Here is one implementation.

Upvotes: 0

Related Questions