CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquerydependenciesgoogle-cloud-functionsgoogle-cloud-storage
Bava
Bava

Reputation: 73

Google Cloud Function - Error: Cannot find module google-cloud/bigquery

I am working to create a cloud function that moves avro files from GCS to BigQuery, anytime a new file lands in GCS. I am using the cloud function ui in GCP. I have 512 MB for memory allocated. Trigger is Google Storage. Event Type if Finalize/Create. Source Code is Inline Editor.

Below is my code. I can successfully deploy, but I am receiving the below error post deployment, and nothing moves to BigQuery.

Additionally, I am attempting to move avro files from a folder WITHIN a bucket, so I am not pulling directly from the top parent bucket. That is the purpose of the below code, as I attempt to get into the folder, which is called "example_spend/"

error: Cannot find module google-cloud/bigquery

'use strict';

exports.createExampleTableFromFile = function(event, callback) {
const file = event.data;
if (file.resourceState === 'exists' && file.name && 
    file.name.indexOf('example_spend/') !== -1) {

console.log('Processing file: ' + event.data.name);


const BigQuery = require('@google-cloud/bigquery');
const Storage = require('@google-cloud/storage');
const assert = require('assert');


 const filename = event.data.name;
 const bucketName = event.data.bucket;


const projectId = "gcp-pilot-192921";
const datasetId = "example_etl";
const tableId = filename.slice(0,filename.indexOf(".")).toLowerCase();  


const bigquery = new BigQuery({
  projectId: projectId,
});

const storage = Storage({
  projectId: projectId
});  


const metadata = {
  sourceFormat: 'AVRO', 
  autodetect: true, 
  writeDisposition: 'WRITE_TRUNCATE'
};

bigquery
  .dataset(datasetId)
  .table(tableId)
  .load(storage.bucket(bucketName).file(filename), metadata)
  .then(results => {
    const job = results[0];


    assert.equal(job.status.state, 'DONE');
    console.log(`Job ${job.id} to load table ${tableId} completed.`);  


    const errors = job.status.errors;
    if (errors && errors.length > 0) {
      throw errors;
    }

  })
  .catch(err => {
    console.error('Error during load job:', err);
  });

callback();
}};

Upvotes: 1

Views: 3073

Answers (1)

Ben P
Ben P

Reputation: 3379

It looks like you haven't added any dependencies to your function:

When using the inline editor, click on to "requirements.txt" for python or "package.json" for javascript, where you can enter the required packages that your function needs to run, these will then be imported when your function spins up. Note that you can also specify versions if required, for example in python: requests==2.19.0.

enter image description here

Upvotes: 3

Related Questions