CODEWITHSUNDEEP
mongodbreactive-streamschangestream
Sayeed
Sayeed

Reputation: 39

Change Stream Duplication if Microservices instance got replicated

I have implemented MongoDB Change Stream in a Java Microservice, When i do a replica of my Microservice I See change stream watch is listening twice. Code is duplicated. Any way to stop this?

Upvotes: 2

Views: 1344

Answers (1)

Alec Henninger
Alec Henninger

Reputation: 370

I gave a similar answer here, however as this question is directly related to Java, I feel it is actually more relevant on this question. I assume what you are after is each change being processed only once among many replicated processes.

Doing this with strong guarantees is difficult but not impossible. I wrote about the details of one solution here: https://www.alechenninger.com/2020/05/building-kafka-like-message-queue-with.html

This solution is implemented in a proof-of-concept library written in Java that accomplishes this which you are free to use/fork (the blog post explains how it works).

It comes down to a few techniques:

  • Each process attempts to obtain a lock
  • Each lock (or each change) has an associated fencing token
  • Processing each change must be idempotent
  • While processing the change, the token is used to ensure ordered, effectively-once updates.

More details in the blog post.

Upvotes: 1

Related Questions