CODEWITHSUNDEEP
firebasegoogle-cloud-functions
FredyC
FredyC

Reputation: 4067

How to correctly handle idempotency of Firebase Cloud Functions?

I have seen all these questions and resources, but neither can satisfy my concerns.

Cloud functions and Firebase Firestore with Idempotency

Firebase cloud function idempotency in docs

How to make idempotent aggregation in Cloud Functions?

https://cloud.google.com/functions/docs/bestpractices/retries#make_retryable_background_functions_idempotent

If a function is executed more than once and both run at the same time, how can I know another one is running and safely discard it? Even if I write eventId somewhere to DB, there is still a chance the other function will run before I manage to write such information.

Can using Firestore transaction help with that? If I would write to a document based on eventId inside the transaction, is it safe to say any other function will hold till I release "the lock"?

The approach of using .set is not viable to me because some functions are generating a unique ID for documents so I would end up writing those multiple times anyway. Some other functions are pretty elaborate and generating (or rather transforming) a bunch of documents at the same time.

As much as I like the whole idea of cloud functions, I wish there was more of the baked-in solution for this. Why can't Firebase do basically what they want from developers? They could write eventId into some internal DB and prevent running function multiple times. Sounds really silly forcing everyone to handle it on their own.

Upvotes: 3

Views: 1475

Answers (1)

Doug Stevenson
Doug Stevenson

Reputation: 317497

Idempotency is difficult, and there is no One Correct Way to implement it. There are lots of options, and different situations to consider. On top of that, not all functions need to be idempotent, so it would be expensive and unnecessary to force it on all implementations. You have to use your best judgement, based on what the function has to do.

Bear in mind that idempotency is not the same as parallelism. You don't need to worry about a function being invoked twice at the same time with the same event. There's no need to worry about "locking" anything to prevent this from happening. All you need to worry about is making sure that a second invocation of a function doesn't do anything incorrect beyond what the first successful invocation would do.

For most functions, simply recording that a particular event ID happened successfully is enough. Future invocations can simply check to see if the event has been handled, and terminate early if so. You can store this wherever you want. Use a transaction if you want, to be safe. You should probably retain that record for at least a few hours.

Bear in mind that functions that perform multiple items of work will be increasingly difficult to make idempotent, as you might have to record whether or not each stage of work was completed successfully. Given the increasing level of difficulty, it's best if a function only commits a single change, then passes the result on to another function, maybe via a pubsub trigger with a payload that indicates what should happen next.

As you can see, it's not simple, and you have to think through it to handle things in a way that suits the requirement of your system.

Upvotes: 13

Related Questions