CODEWITHSUNDEEP
firebasegoogle-app-enginegoogle-cloud-functionsgoogle-cloud-storagefirebase-storage
Parakoos
Parakoos

Reputation: 1323

How to create long-running user data export for a Firebase app?

I have a Firebase app that uses Firestore as the database, Storage to store user's photos and Functions for various things. I want to be a good citizen and give my users a way to zip up all their data and download it, if they decided to leave me one day.

My plan is to do this:

  1. Create a Zipped file write stream to a export Storage Bucket
  2. Loop over all the user's Storage files and add them to the Zipped output stream.
  3. Loop over the user's Firestore data, turn it into a JSON doc and add it to the zipped output stream.
  4. Email the user with a download link for the zipped file.

But this is where I seem to hit the limitations of what Functions can do. After all, this export can take much longer than 9 minute time limit of a Functions call if I have years worth of photos to export.

What is the best solution for this problem?

I don't want to answer my own question, but having done my research, it seems that perhaps an option would be to add Google Cloud Tasks and Google Cloud AppEngine to my application architecture, but since neither is mentioned in the standard Firebase Console, I'm hesitant as they don't seem to be part of the 'Firebase vision'.

Upvotes: 0

Views: 101

Answers (1)

Thomson Tran
Thomson Tran

Reputation: 189

You will have to add App Engine to your project, seeing as the 9 minute duration limit can't be increased [1]

App Engine is usually added to Firebase projects when you need to do additional backend logic [2] [3]

[1]https://cloud.google.com/functions/quotas#time_limits

[2]https://cloud.google.com/solutions/mobile/mobile-app-backend-services#matrix

[3]https://firebase.google.com/docs/storage/gcp-integration#google_app_engine

Upvotes: 1

Related Questions