CODEWITHSUNDEEP
javamongodbmongodb-javachangestream
Konrad Lötzsch
Konrad Lötzsch

Reputation: 478

What is the java driver behaviour, when the resume token for mongodb change streams is outdated?

When watching a change stream on a collection, I can specify a resume Token (with resumeAfter) to get changes after that token in the opslog. What happens, if this token cannot be found in the opslog (for example, the client was disconnected too long and provides an old token)?

Especially, will the async mongodb java driver detect this situation and how?

For me, that question is important because i must detect, whether I lost the connection and have to pull all data again or can rely on the changestream to get all changes from the collection.

The documentation states, that

Change streams are resumable, as long as the oplog has enough history to locate the last operation that the application received.

but does not specify, what happens otherwise. I also find it rather difficult to read the driver source, to determine how a client can detect whether the last token could be found in the opslog.

Upvotes: 0

Views: 1556

Answers (1)

Wan B.
Wan B.

Reputation: 18845

What happens if this token cannot be found in the oplog

As mentioned on the comment, this behaviour happens in the MongoDB server. When a client (i.e. application written using the Java driver) submits a resumeToken the server will validate the token. See also resume_token.cpp (v3.7.0)

Any errors found by the server will be thrown back to the client, the client will then raise an exception. In the case of MongoDB Java driver it would be com.mongodb.MongoCommandException.

If falling off of the OpLog is a concern, I would suggest to calculate the replica set oplog size accordingly in conjunction with the caching period of the resume token. For example. If the OpLog could store up to 24 hours of operations, the resume token may be cached every 8-12 hours.

Upvotes: 1

Related Questions