CODEWITHSUNDEEP
node.jsfirebasegoogle-cloud-firestore
gbalduzzi
gbalduzzi

Reputation: 10186

Listen to realtime updates in Firestore without downloading all initial data

I developed a node script that listens to the changes on a firestore collection and forward them to a REST service:

const db = admin.firestore();
const collectionRef = db.collection('users');

var collObserver = collectionRef.onSnapshot(querySnapshot => {
        let changes = querySnapshot.docChanges();

        changes.forEach(change => {
           console.log("Change", change.type, change.doc.id, change.doc.data())
           // Do stuff with changes
        });
    }, err => {
        console.log(`Encountered error: ${err}`);
    }
);

However, when the script loads the onSnapshot method trigger an initial request for the full collection. This is a problem, because my collection may contain up to million of records and there is no need for the script node to download and manage all of them, it should only get the changes.

I don't see any option for that in the official SDK. Are there some hacks/solutions for the problem?

Upvotes: 3

Views: 556

Answers (1)

Doug Stevenson
Doug Stevenson

Reputation: 317808

Queries listeners always receive all matching documents that would be matched by the query. You can't change that. The only thing you can change is your query. Currently, your query is asking for all documents in the collection, and you will have to narrow that down with a filter.

What you can do instead is make sure your query only requests as much data as it wants. If your documents have a natural time ordering, try attaching a timestamp to each of the documents, and querying for only the documents that are new since the last time you performed the query. If you don't have some sort of "checkpoint" to figure out what's new vs old, then you'll have a hard time making this work out.

Upvotes: 3

Related Questions