How to import a .json file to Firebase Database?

Question

I tried a few times via the web UI, but first the file wasn’t shown to me in the explorer and when i finally got around that and uploaded my .json there was one entry:

error: "Auth token is expired“

i just want to import that .json file, how does one do that?

Answer 1

I have successfully imported data from a JSON file into Firebase by running a Node.js script.

Pre-requesites: Before running this script, ensure that the following packages are installed either globally or available within your project.

I utilized the following packages:

• npm i firestore-export-import
• npm i firebase-admin

Packages short description:

1. firestore-export-import- lets you export and import data from firestore with sub collection;
2. fs - File system package, aiding in file read/write operations;
3. path - module provides utilities for working with file and directory paths. (I used this to get directory separator by getting path.sep);
4. firebase-admin - enables access to Firebase services from privileged environments (such as servers or cloud) in Node.js.

Your serviceAccount.json file [https://stackoverflow.com/questions/40799258/where-can-i-get-serviceaccountcredentials-json-for-firebase-admin)

{
  "type": "service_account",
  "project_id": "yourProject_id",
  "private_key_id": "yourPrivate_key_id",
  "private_key": "yourPrivate_key",
  "client_email": "yourClient_email",
  "client_id": "yourClient_id",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/yourClient_x509_cert_url",
  "universe_domain": "googleapis.com"
}

Here is an example of what this actually does: it removes IDs and adds them as keys to those objects where they exist, or simply adds a key if there was no ID in the object

how it looks like in firebase after import:

end result - code gist.github

// Imports
const firestoreService = require('firestore-export-import')
const serviceAccount = require('./serviceAccount.json')
const fs = require('fs')
const path = require('path')
const tempFileName = `${__dirname}${path.sep}data-temp.json`;
const admin = require('firebase-admin');
// Initialize Firebase Admin SDK
admin.initializeApp({
  credential: admin.credential.cert(serviceAccount)
});

// Get a reference to the Firestore database
const db = admin.firestore();
(async() => {
  try {
    const fileContents = fs.readFileSync(`${__dirname}/src/data.json`, 'utf8');
    const data = JSON.parse(fileContents);
    const transformed = transformDataForFirestore(data);
    fs.writeFileSync(tempFileName, JSON.stringify(transformed));
    await jsonToFirestore();
    fs.unlinkSync(tempFileName);
  } catch (error) {
    console.error('Error occurred during the import process:', error);
  }
})();
async function jsonToFirestore() {
  try {
    console.log('Initialzing Firebase')
    await firestoreService.initializeFirebaseApp(serviceAccount)
    console.log('Firebase Initialized')

    await firestoreService.restore(db, tempFileName)
    console.log('Upload Success')
  } catch (error) {
    console.log(error)
  }
}

// In order to preserve ids in data.json
// as ids in firestore
// must use keyed object (id being the key) instead of array of records
function transformDataForFirestore(data) {
  const collections = { ...data
  }; // Create a shallow copy to avoid mutating the original 'data'
  delete collections.stats;

  const collectionsById = {};

  Object.keys(collections).forEach((collectionKey) => {
    collectionsById[collectionKey] = {};

    // Check if 'collection' is an array before iterating
    const collection = collections[collectionKey];
    if (Array.isArray(collection)) {
      collection.forEach((record) => {
        collectionsById[collectionKey][record.id] = { ...record
        }; // Create a copy of 'record' to avoid modifying the original object
        delete collectionsById[collectionKey][record.id].id;
      });
    } else {
      console.error(`The collection '${collectionKey}' is not an array.`);
      // Decide how to handle this scenario, whether to skip or handle differently
    }
  });

  return collectionsById;
}

Parts which can be improved:

• using deep cloning with lodash library, more on that here

Parts of this code was taken from this source - gist.github

Answer 2

The Auth Token probably expired because you were on the page too long before you uploaded it. Keep trying and it will work, given that the .json is correctly formatted.