CODEWITHSUNDEEP
firebasefirebase-realtime-databasegoogle-cloud-firestore
Superian007
Superian007

Reputation: 395

Sorting data while using distributed counter

In the Firestore guide it's mentioned that one should use distributed counters to handle multiple updates of a single document at the same time.

This got me thinking. With this counter implementation how can you sort your data? For example let's say that I have a database of posts and each post has its own distributed counter for upvotes. Now, I'd like to get maybe 20 most upvoted posts. How can you do that?

From my understanding of Firestore database this can't be done. Unless I'm missing something? Is there a better solution for counters for such use case I described above?

Upvotes: 2

Views: 1132

Answers (2)

leftL
leftL

Reputation: 168

According to this post, an alternative option is to use realtime database to replace the distributed counter on firestore, and still need to periodically read the counts from realtime database and write it back to firestore.

Not sure if this is a more scalable way than distributed counter.

Upvotes: 0

Frank van Puffelen
Frank van Puffelen

Reputation: 598775

The main problem with counters is this limit:

In Cloud Firestore, you can only update a single document about once per second, which might be too low for some high-traffic applications.

The same documentation page that explains how to use distributed counters to work around this, also shows an example of how to then read the counter totals:

To get the total count, query for all shards and sum their count fields:

function getCount(ref) {
    // Sum the count of each shard in the subcollection
    return ref.collection('shards').get().then(snapshot => {
        let total_count = 0;
        snapshot.forEach(doc => {
            total_count += doc.data().count;
        });

        return total_count;
    });
}

It also mentions the main disadvantage of this:

Cost - The cost of reading a counter value increases linearly with the number of shards, because the entire shards subcollection must be loaded.

One way to work around this, is to periodically read the counts from the shards and update a master list of counts. This essentially turns the whole exercise into a map-reduce solution. You'll want to run this reduce-code on a schedule, since otherwise, you'd still run into the write rate limit. Using a periodically triggered Cloud Function sounds ideal for this.

Upvotes: 6

Related Questions