CODEWITHSUNDEEP
firebasefirebase-realtime-databasegoogle-cloud-functions
Glenn
Glenn

Reputation: 2206

How to deal with firebase trigger function execution order

I have a firebase http function which appends data to a firebase database list. A second function is configured to process the list when it changes and then update some summary data. In my case, these updates come in bursts. I'm using node.js firebase functions.

Looking at the firebase logs, I see this sequence when starting with an empty list:

  1. add to list from http - list has 1 element
  2. add to list from http - list has 2 elements
  3. add to list from http - list has 3 elements
  4. add to list from http - list has 4 elements
  5. summarize list with 1 element
  6. summarize list with 3 elements
  7. summarize list with 4 elements
  8. summarize list with 2 elements

My problem is the summary only includes 2 elements instead of 4.

It would appear the summarize trigger functions are invoked in parallel instead of sequential so when several trigger in close proximity, the last one to finish may be be one of the first ones triggered instead of the last.

What approaches can be used to insure the summary computation has 'all the data' and a prior summary computation running slower doesn't overwrite a later one? Can firebase function triggers be serialized to execute in the order they are initiated?

Ideally I'd like to avoid computing the summary N times when a burst comes in so some solution where the summary could be 'scheduled' for some short time in the future and then canceled and rescheduled if a new event comes in would be nice.

Upvotes: 4

Views: 3157

Answers (2)

Doug Stevenson
Doug Stevenson

Reputation: 317968

There's absolutely no guarantee for the order of delivery for events coming from multiple clients or invocations. In fact, you'll have a hard time even defining the timing of the events, since there's a lot of variable moving parts between the moment a client makes a request, and the moment the final work inside your function is complete for that client.

The best thing you can do is assume that multiple clients are all effectively sending out-of-order requests to your function, and use database transactions to prevent any sort of collision for the writes they make.

If you absolutely must serialize things, you'll need to have some other program or agent define the correct sequences and serialize all the work, ensuring that all writes happen in a predictable sequence.

If you want to read an extended discussion about why ordering is difficult, read this article (Cloud Functions is built on top of pubsub).

Upvotes: 4

Glenn
Glenn

Reputation: 2206

My workaround is to store a admin.database.ServerValue.TIMESTAMP with the list add and verify in the results calculator that it produced results for the latest timestamp. If not, it tries again. In most cases it will not need to recompute the summary as my input source is normally sporadic single list adds rather than lumped additions. I implemented this as a function returning a Promise that calls itself if necessary to recompute. This is the sequence:

  1. Read current list and timestamp
  2. Compute summary results and store them
  3. Read the timestamp again
  4. If timestamp different, go to 1, else done

Here's the code:

/// return a Promise that new summary and detail results will be posted
function updateResults(regattaId, lapdataTS, depth) {
  if (depth > 10) {
    return Promise.reject("Too many recomputes");
  }
  return admin.database().ref('/eventdata/'+regattaId).once('value')
  .then(function (snapshot) {
    const rawdata = snapshot.val(); 

    if (rawdata.lapdataTS === lapdataTS) {
        // console.log("already computed");
        return Promise.resolve();
    }
    lapdataTS = rawdata.lapdataTS ? rawdata.lapdataTS : null;
    const results = regattaCalc.computeResults(rawdata);

    var updates = {};
    updates['results/' + regattaId] = results;
    updates['summary/' + regattaId] = results.regattaInfo;
    return admin.database().ref().update(updates);
  }).then(function () {
    // read last TS and see if it matches our summary
    return admin.database().ref('/eventdata/'+regattaId+'/lapdataTS').once('value');
  }).then(function (snapshot) {
   if (snapshot.val() === lapdataTS) {  
       return Promise.resolve();
   } else {
       //console.log("Need to calc again");
       return updateResults(regattaId, lapdataTS, depth++);
   }
  }).catch((reason) => {
    console.log("Error generating summary: " + reason);
    return Promise.reject(reason);
  });
}

exports.compupteResults = functions.database.ref('/eventdata/{regattaId}').onWrite(event => {
return updateResults(regattaId,null,0);
 });

Upvotes: 1

Related Questions