CODEWITHSUNDEEP
javascriptnode.jsgoogle-cloud-platformgoogle-bigquerygoogle-cloud-functions
mauricioSanchez
mauricioSanchez

Reputation: 366

Issues loading data into a BigQuery table using a cloud function

I'm having issues trying to load a Data store backup into an existing table in BigQuery. I'm getting the following error:

TypeError: bigquery.dataset(...).table(...).load is not a function

I'm following one of the examples in the BigQuery cloud Api repo.

Not sure if I'm using this the wrong way, but what I'm trying to achieve is just have my cloud function update this BigQuery table from a Data Store dump, as opposed to delete and create daily - which seems counter productive and I was doing before.

Here is my code:

exports.processFile = function (event, callback) {

  const BigQuery = require('@google-cloud/bigquery');
  const Storage = require('@google-cloud/storage');

  const bucketName = 'nyt-links-backup-dev';
  const filename = event.data.name;
  const tableId = 'links_data_tbl';
  const projectId = 'nyt-sartre-dev';

  // Instantiates clients
  const bigquery = new BigQuery({
    projectId: projectId,
  });

  const storage = new Storage({
    projectId: projectId,
  });

  const datasetId = 'sartre_sublink_dataset';
  const dataset = bigquery.dataset(datasetId);

  const metadata = {
    sourceFormat: 'AVRO',
  };     

  // Loads data from a Google Cloud Storage file into the table
  bigquery
    .dataset(datasetId)
    .table(tableId)
    .load(storage.bucket(bucketName).file(filename), metadata)
    .then(results => {
      const job = results[0];

      // load() waits for the job to finish
      assert.equal(job.status.state, 'DONE');
      console.log(`Job ${job.id} completed.`);

      // Check the job's status for errors
      const errors = job.status.errors;
      if (errors && errors.length > 0) {
        throw errors;
      }
    })
    .catch(err => {
      console.error('ERROR:', err);
    });
    callback();
};

Upvotes: 0

Views: 1211

Answers (1)

Kat
Kat

Reputation: 1654

Sounds like you are using a version of the BigQuery library that doesn't have the load function on Tables. The load function was introduced in December 2017 in version 0.12.0 and was previously called import.

Upvotes: 2

Related Questions