CODEWITHSUNDEEP
node.jsfirebasecsvgoogle-cloud-firestoregoogle-cloud-functions
Thomas Smeman
Thomas Smeman

Reputation: 195

CSV file from firebase storage with nodeJS

I need to get a CSV file from firebase storage, then parse it so I can send data to the right documents in firestore. I had a working function, but I don't know why. This is the code that should parse the CSV data. Currently, I get undefined in the data constant, so I think the data from the csv gets converted to utf8 incorrectly. However, it used to run before.

exports.importCSVAppointments = functions.region('europe-west3').storage.object().onFinalize((object) => {
  const name = object.name;
  //Test if file is in correct folder and is csv file
  const nameSplit = name.split('/');
  const organisation = nameSplit[1].split('_')[1];
  const folder = nameSplit[0];
  if(folder == 'imports'){
    //Retrieve the file from firebase storage
    const storage = new Storage();
    const bucket = storage.bucket('gs://fmis-online-dev.appspot.com');
    admin.storage().bucket().file(name)
      .download(function (err, contents) {
          if (!err) {
            const convert = (from, to) => str => Buffer.from(str, from).toString(to);
            const hexToUtf8 = convert('hex', 'utf8');
            const data = hexToUtf8(contents);
            const dataSplit = data.split('\n');
            console.log(dataSplit);
            for(i=1;i<=dataSplit.length-1;i++){
              console.log('actually runs');
              //This will run for every appointment

              //Get appointment data
              const text = dataSplit[i];
              const parsedData = text.match( /\s*(\".*?\"|'.*?'|[^,]+)\s*(,|$)/g ).map( function (text) {
                let m;
                if (m = text.match(/^\s*\"(.*?)\"\s*,?$/)) return m[1]; // Double Quoted Text
                if (m = text.match(/^\s*'(.*?)'\s*,?$/)) return m[1]; // Single Quoted Text
                if (m = text.match(/^\s*(true|false)\s*,?$/)) return m[1] === "true"; // Boolean
                if (m = text.match(/^\s*((?:\+|\-)?\d+)\s*,?$/)) return parseInt(m[1]); // Integer Number
                if (m = text.match(/^\s*((?:\+|\-)?\d*\.\d*)\s*,?$/)) return parseFloat(m[1]); // Floating Number
                if (m = text.match(/^\s*(.*?)\s*,?$/)) return m[1]; // Unquoted Text
                return text;
              } );
              console.log(parsedData);

This is the CSV file that needs to be parsed. The first row is always the title

subject,contactName,contactEmail,contactPhoneNumber,dateStart,dateEnd,timeStart,timeEnd,isRecurring,frequency,days,location,room,visitorType,visitorName,visitorPhoneNumber,visitorEmail,licensePlate,sendInvite
Test CSV,name,[email protected],+31 6 12345678,26-2-2021,26-2-2021,12:00,13:00,no,,"[]",Katrien NV,2A,Cursist,Dr. Doofenschmirtz,+31 6 98765432,[email protected],,no

Upvotes: 0

Views: 1028

Answers (1)

Renaud Tarnec
Renaud Tarnec

Reputation: 83093

Since this code is executed in a Cloud Function, you should use the "promise version" of the asynchronous download() method, as follows:

  if (folder == 'imports') {
    //Retrieve the file from firebase storage
    const storage = new Storage();
    const bucket = storage.bucket('gs://fmis-online-dev.appspot.com');
    return admin.storage().bucket().file(name). // <==!!! see the return here
      .download()
      .then(data => {
          const contents = data[0];
  
          // ....
          
          return null;   // <==!!! see the return here
      })
      .catch(error => {
         // ....
          return null;    // <==!!! see the return here
      })
   } else {
      return null;    // <==!!! see the return here
   }

You'll find in the doc the explanation on why it is important to return a Promise, in order to correctly manage the life cycle of the Cloud Function.

You don't show how you write to Firestore, but you must include in the Promises chain the Promises returned by the asynchronous Firestore methods, e.g.:

    return admin.storage().bucket().file(name). // <==!!! see the return here
      .download()
      .then(data => {
          const contents = data[0];
  
          // ....
          const parsedData = ....

          return admin.firestore()   // <==!!! see the return here
          .doc('xxx/yyyy').set( { parsedData, foo: bar });
      })
      .catch(error => {
         // ....
          return null;    // <==!!! see the return here
      })

Upvotes: 2

Related Questions