CODEWITHSUNDEEP
apache-kafkaquarkussmallrye
Shady Ragab
Shady Ragab

Reputation: 725

lost messages in quarkus service conumes kafka topic

I have 30 partitions in kafka topic, during my quarkus service is up and running, if the service consumed for example from (5) partitions, whatever messages are written to these partitions during the service is down will be consumed after the service restart, whatever written to other partitions(25) during the service is down will not be consumed, how to fix this behavior?

another observation, in kafka ui, during the service is running, (30) partitions are assigned, when it's down, only number of partitions service consumed messages from them before are assigned (5)

how to fix this behavior so that I have no lost messages ?

technologies: java 17 - quarkus - smallrye - kafka

Note : because it could be relevant to the issue : I have this property in my config auto-offset-reset: earliest and it does not work as expected, I mean the preexisting messages are not consumed, only new messages after the service with new consumer group starts are consumed

other may be only relevant configuration:

enable-auto-commit: false

@Acknowledgment(Acknowledgment.Strategy.POST_PROCESSING)

Upvotes: 0

Views: 40

Answers (0)

Related Questions