CODEWITHSUNDEEP
javascriptfirebasefirebase-realtime-databasegoogle-cloud-functions
Tnargib
Tnargib

Reputation: 73

Limit number of records in firebase

Every minute I have a script that push a new record in my firebase database.

What i want is delete the last records when length of the list reach a fixed value.

I have been through the doc and other post and the thing I have found so far is something like that :

// Max number of lines of the chat history.
const MAX_ARDUINO = 10;

exports.arduinoResponseLength = functions.database.ref('/arduinoResponse/{res}').onWrite(event => {
  const parentRef = event.data.ref.parent;
  return parentRef.once('value').then(snapshot => {
    if (snapshot.numChildren() >= MAX_ARDUINO) {
      let childCount = 0;
      let updates = {};
      snapshot.forEach(function(child) {
        if (++childCount <= snapshot.numChildren() - MAX_ARDUINO) {
          updates[child.key] = null;
        }
      });
      // Update the parent. This effectively removes the extra children.
      return parentRef.update(updates);
    }
  });
});

The problem is : onWrite seems to download all the related data every time it is triggered.

This is a pretty good process when the list is not so long. But I have like 4000 records, and every month it seems that I screw up my firebase download quota with that.

Does anyone would know how to handle this kind of situation ?

Upvotes: 1

Views: 2261

Answers (1)

Tnargib
Tnargib

Reputation: 73

Ok so at the end I came with 3 functions. One update the number of arduino records, one totally recount it if the counter is missing. The last one use the counter to make a query using the limitToFirst filter so it retrieve only the relevant data to remove.

It is actually a combination of those two example provided by Firebase : https://github.com/firebase/functions-samples/tree/master/limit-children https://github.com/firebase/functions-samples/tree/master/child-count

Here is my final result

const MAX_ARDUINO = 1500;

exports.deleteOldArduino = functions.database.ref('/arduinoResponse/{resId}/timestamp').onWrite(event => {
    const collectionRef = event.data.ref.parent.parent;
    const countRef = collectionRef.parent.child('arduinoResCount');


    return countRef.once('value').then(snapCount => {
        return collectionRef.limitToFirst(snapCount.val() - MAX_ARDUINO).transaction(snapshot => {
            snapshot = null;
            return snapshot;
        })
    });
});



exports.trackArduinoLength = functions.database.ref('/arduinoResponse/{resId}/timestamp').onWrite(event => {
    const collectionRef = event.data.ref.parent.parent;
    const countRef = collectionRef.parent.child('arduinoResCount');

    // Return the promise from countRef.transaction() so our function 
    // waits for this async event to complete before it exits.
    return countRef.transaction(current => {
        if (event.data.exists() && !event.data.previous.exists()) {
            return (current || 0) + 1;
        } else if (!event.data.exists() && event.data.previous.exists()) {
            return (current || 0) - 1;
        }
    }).then(() => {
        console.log('Counter updated.');
    });
});


exports.recountArduino = functions.database.ref('/arduinoResCount').onWrite(event => {
    if (!event.data.exists()) {
        const counterRef = event.data.ref;
        const collectionRef = counterRef.parent.child('arduinoResponse');

        // Return the promise from counterRef.set() so our function 
        // waits for this async event to complete before it exits.
        return collectionRef.once('value')
            .then(arduinoRes => counterRef.set(arduinoRes.numChildren()));
    }
});

I have not tested it yet but soon I will post my result !

I also heard that one day Firebase will add a "size" query, that is definitely missing in my opinion.

Upvotes: 1

Related Questions