CODEWITHSUNDEEP
javascriptnode.jsasynchronousgoogle-cloud-storagegoogle-cloud-functions
Bogac
Bogac

Reputation: 3707

Google Cloud Storage (Js), Async/Await & Cloud Functions Errors

THIS IS SOLVED ON GOOGLE SIDE NOW

Google Cloud NodeJs library now fix integrated. Keeping this question just for reference.

ORIGINAL QUESTION

I wanted my code to be clean(er) and used Typescript & async/await writing Cloud Functions accessing Google Cloud Storage.

Some important parts of my tsconfig.json:

{
  "compilerOptions": {
    "declaration": false,
    "target": "es6",
    "module": "commonjs",
    "noImplicitAny": true,
    "removeComments": true,
    "noUnusedLocals": true,
    "moduleResolution": "node",
    "sourceMap": false,
    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2016"
    ],
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "outDir": "./"
  }
}

First error I had was ECONNRESET and I was making a call like this which probably caused it:

await bucket.file(path).download({
  destination: tempFilePath
})

I've figured rest of the function did not wait till this line to be finished and whole function execution ends before file is downloaded from GCS to temp path. So I've put that section into try/catch block:

  try { 
    await bucket.file(path).download({
      destination: tempFilePath
    })
  } catch (error) {
    console.log(error);
  }

It worked fine till today. Today I had this error:

convert: Empty input file '/temp/img-file.jpg'

which again made me think that next line (converting image size) is executed before file is downloaded from bucket to temp folder.

Am I missing something ?

Upvotes: 1

Views: 3021

Answers (1)

Bogac
Bogac

Reputation: 3707

Apparently I've made some errors due to fact that there are still many libraries out there not having proper type definitions, or not at all, including Google ones, and me making wrong guesses.

Be careful using libraries without Type Definitions and open their source to see return types.

While searching a solution I ran into this ansfer as well: GCS permissions related error and a bash script to fix. If you read this answer maybe your problem is related to that, so I share it here too.

UPDATE: Again I've seen an error, so to double make sure I've included both async calls in try/catch:

  let myBucket : any = {};

  try {
    myBucket = await googlecloudstorage.bucket('<bucket-id>');
    await myBucket.file(path).download({
      destination: tempFilePath
    })
  } catch (error) {
    console.log(error);
  }

UPDATE 2: Still having errors, trying o figure it out within Typescript but I plan to go back to plain javascript. Also I've tried updating ACLs with a batch file as explained in issue I've provided link above, didn't seem to help.

UPDATE 3 (Hopeful to be last): Well, for calls related to Google Cloud Storage I use plain javascript promises now and ditch async/await, like ones used in Firebase-Functions Samples repo, so far so good.

UPDATE 4: I kept getting errors, after changing my code, upon uploads/reads inconsistently. Made further searches and...

Read this answer from another question describing same problem. Basically it's not us, it's them, but they don't admit. It's because the way google-cloud-node package uses sockets. Will post it here if some sane way of fixing this comes up.

UPDATE 5:

Here is how not to keep sockets open forever:

var gcs = require('@google-cloud/storage')(...)
gcs.interceptors.push({
  request: function(reqOpts) {
    reqOpts.forever = false
    return reqOpts
  }
})

...seems to work, for now.

Upvotes: 3

Related Questions