CODEWITHSUNDEEP
mongodbchangestream
erezarnon
erezarnon

Reputation: 126

Mongo changestream timestamp

When tailing the oplog, I see a timestamp for each event. Change streams have advantages over tailing the oplog directly, so I'd like to use those. However, I can't find any way of figuring out when a change occurred. This would be problematic if my script went down for a while and then resumed using a resume token.

Is there any way of getting that timestamp?

Upvotes: 1

Views: 2289

Answers (1)

Wan B.
Wan B.

Reputation: 18835

I can't find any way of figuring out when a change occurred.

Currently (MongoDB v3.6), there is no way to find out the timestamp of an event returned by the server from the receiving end. This is because the cluster time's timestamp is actually embedded into the resume token as a binary format.

There is a ticket to request adding a tool to inspect this resume token SERVER-32283. Feel free to watch/upvote for updates on the ticket.

This would be problematic if my script went down for a while and then resumed using a resume token.

When resuming Change Streams using the resume token, it will resume from that point forward. This is because the token contains the cluster time, and the server is aware when the last operation the token has 'seen'.

You also said down for a while. Change streams is build upon Replica Set Oplog, which also means the resumability nature of change streams is limited by the size of the oplog window.

For example, if the last cached token time was 24 hours ago and the oplog size is only 12 hours, your application will not be able to utilise the change streams resume token. Since you're comparing change streams with tailing the oplog, in this regard both would have had the same potential issue.

If this is a real concern for your use case, please adjust your oplog window size accordingly. i.e. if the receiving client will have a potential downtime greater than the oplog window time.

See also Change Streams Production Recommendations

Upvotes: 2

Related Questions