Migrating Nested Data from the Realtime Database to Firestore

Question

I´m migrating a Firebase Realtime Database to Firestore and I have nested data

and I have nested data from which I want to create a collection.

Example:

"data" : {
  "-LYBzlXPoN0137KRLovk" : {
    "-LYC-HHqDFgL9PovJiBr" : {
      "age" : 35,
      "country" : "Country",
      "date" : "2019-02-08T13:07:10+01:00",
      "gender" : "male",
      "id" : 1549627467620,
    },
    "age" : 35,
    "country" : "Country",
    "date" : "2019-02-08T13:04:27+01:00",
    "gender" : "male",
    "id" : 1549627467620,

I want to create a subcollection from the nested array in Firestore. Therefore I tried the following (Cloud Function):

exports.migrateVisits = functions.database.ref('/data/{key}/{nestedKey}')
    .onWrite((change:any, context:any) => {
        // Get a reference to the Firestore document of the changed user
        let userDoc = admin.firestore()
            .collection(`data/{key}/nestedKeys`)
            .doc(context.params.nestedKey);
        // If this user has been deleted, delete in Firestore also
        if (!change.after.exists()) {
            return userDoc.delete();
        }
        // Get the user object with the new changes,
        // as opposed to its value before the edit
        let userData = change.after.val();
        // Now update Firestore with that change

        return userDoc.set(userData);
    });

But I get the error: Error: Argument "data" is not a valid Document. Input is not a plain JavaScript object.

Do you have any idea to migrate in the best way nested subdata?

Update for Renaud

exports.migrateVisits = functions.database.ref('/data/users/{userId}/visits/{visit}')
    .onWrite((change:any, context:any) => {
        let userData = change.after.val();
        const mainDocObj:any = {};

        let batch = admin.firestore().batch();

        Object.keys(userData).forEach(e => {
            console.log(e);
            if (
                e == 'age' ||
                e == 'country' ||
                e == 'date' ||
                e == 'gender' ||
                e == 'id' ||
                e == 'lastVisit' ||
                e == 'magazineRoute' ||
                e == 'magazineRouteDone' ||
                e == 'magazineRouteLastDate' ||
                e == 'name' ||
                e == 'notice' ||
                e == 'onNextVisit' ||
                e == 'pause' ||
                e == 'placements' ||
                e == 'plz' ||
                e == 'street' ||
                e == 'tag' ||
                e == 'type'
            ) {  //here, add the other main keys, e.g. with ['gender', 'age', 'country', ....].includes(e)
                mainDocObj[e] = userData[e];
            } else {
                //it needs to be added as a doc in the sub-collection
                const subDocRef = admin
                    .firestore()
                    .collection(`users/${context.params.userId}/vi/${context.params.visit}/rv`)
                    .doc(e);
                batch.set(subDocRef, userData[e]);
            }
        });

        //We first write the mainDoc
        console.log(mainDocObj);
        return admin
            .firestore()
            .collection(`users/${context.params.userId}/vi`)
            .doc(context.params.visit)
            .set(mainDocObj)
            .then(() => {
                //We then write the children in one batch
                return batch.commit();
            });
    });

Answer 1

The following should do the trick:

exports.migrateVisits = functions.database.ref('/data/{key}/{nestedKey}')
    .onWrite((change:any, context:any) => {
        // Get a reference to the Firestore document of the changed user

        const key = context.params.key;

        const userDoc = admin
          .firestore()
          .collection('data/' + key + '/nestedKeys')              
          .doc(context.params.nestedKey);

        // If this user has been deleted, delete in Firestore also
        if (!change.after.exists()) {
            return userDoc.delete();
        }
        // Get the user object with the new changes,
        // as opposed to its value before the edit
        let userData = change.after.val();
        // Now update Firestore with that change

        return userDoc.set(userData);
    });

You have to use context.params to get values of the path.

---

UPDATE following our comments.

The main point to note is that since we now listen at the level of the /data/{key} you have to distinguish the data that belongs to the main doc from the one that belongs to the children. In the code below I propose to do that based on the data name (gender, date, age, id...). If, while looping, you encounter a data item with another name (e.g. an id like -LYC-HHqDFgL9PovJiBr) it means it is a children doc.

Another point to note is the use of a batch write, see https://firebase.google.com/docs/firestore/manage-data/transactions#batched-writes

Also, I let you take in charge the check if the doc is deleted on not. Another aspect you might have to adapt is the management of children that are already existing, in the case you modify the age under e.g. the node -LYC-HHqDFgL9PovJiBr, because, in this case the trigger will still happens at the level of the /data/{key}.

exports.migrateVisits = functions.database
  .ref('/data/{key}')
  .onWrite((change:any, context:any) => {
    const key = context.params.key;
    let userData = change.after.val();
    const mainDocObj = {};

    let batch = admin.firestore().batch();

    Object.keys(userData).forEach(e => {
      if (e === 'gender') {  //here, add the other main keys, e.g. with ['gender', 'age', 'country', ....].includes(e)
        mainDocObj[e] = userData[e];
      } else {
        //it needs to be added as a doc in the sub-collection
        const subDocRef = admin
          .firestore()
          .collection('data/' + key + '/nestedKeys')
          .doc(e);
        batch.set(subDocRef, userData[e]);
      }
    });

    //We first write the mainDoc
    return admin
      .firestore()
      .collection('data')
      .doc(key)
      .set(mainDocObj)
      .then(() => {
        //We then write the children in one batch
        return batch.commit();
      });
  });