CODEWITHSUNDEEP
cdcdebeziummysqlbinlog
Rajat Goyal
Rajat Goyal

Reputation: 475

Duplicate messages in kafka when using debezium

I read internals of debezium, but it also can send duplicate messages to kafka, if there is some failure at connector just before updating offset in bin log. In that case, consumer needs to be idempotent. Is there any way we can avoid duplicate message in kafka ??

Upvotes: 2

Views: 2096

Answers (1)

mipo256
mipo256

Reputation: 3214

As for now, Debezium aims for at least once delivery only. That actaully means that it is possible that on the consumer side we may, possibly, get duplicates. So the consumer system (in the author's case the Kafka Broker) just needs to be aware of that.

Upvotes: 0

Related Questions