CODEWITHSUNDEEP
javaspring-bootapache-kafkaspring-kafkaspring-scheduled
ranban282
ranban282

Reputation: 187

Intermittently consuming kafka messages in a springboot application

I have the following use-case:

Messages which my application could not process, possibly due to some intermittent issues related to network, etc, are being written to a kafka topic.

I want to periodically read these messages using a cron/scheduled task, and then process them.

So far I've used the KafkaListener in my springboot for my kafka-related needs, and has worked well.

However, I could not find a way to read messages intermittently using KafkaListener.

Is there even a way to do this in spring? Would be glad if someone could link to documents or tutorials or describe a way to achieve what I want to do.

Upvotes: 0

Views: 1738

Answers (2)

Andrey
Andrey

Reputation: 11

Try to use idleBetweenPolls parameter, it will allow you to read periodically

See https://docs.spring.io/spring-kafka/api/org/springframework/kafka/listener/ContainerProperties.html#setIdleBetweenPolls(long)

Upvotes: 1

Alper Derya
Alper Derya

Reputation: 237

You need to manage consuming process manually. It is possible if you disable autostartup feature of KafkaListener.

There are two discussions about this. See below:

Is there any example of Spring Schedule that reads Kafka topic?

Spring Boot Job scheduler with Kafka consumer

Upvotes: 1

Related Questions