CODEWITHSUNDEEP
node.jsgoogle-cloud-platformgoogle-bigquery
Piotr Sobczyk
Piotr Sobczyk

Reputation: 6583

More detailed error messages from Node.js BigQuery client library

I'm using the official Google Node connector to BigQuery.

I have the following snippet to stream records into the database:

module.exports.sendToBigQuery = (rows) => {
    bigquery
        .dataset(DATASET_NAME)
        .table(TABLE_NAME)
        .insert(rows)
        .catch(err => {
            if (err && err.name === 'PartialFailureError') {
                if (err.errors && err.errors.length > 0) {
                    console.log('Insert errors:');
                    err.errors.forEach(err => console.error(err));
                }
            } else {
                console.error('ERROR:', err);
            }
        });
};

Unfortunately, whenever my data doesn't match the scema, all I'm getting is this cryptic error object: { errors: [ { message: 'no such field.', reason: 'invalid' } ],

There is no location field that would tell me which field is missing which makes debugging this code a nightmare for more complex schemas.

Is there any way to enable debug level of errors somehow? Or it's just a bug in client implementation? Any idea how I could access this information?

Upvotes: 2

Views: 1770

Answers (2)

Crispy
Crispy

Reputation: 99

I am not sure what was in the error response at the time of writing but the BigQuery error response now also includes error information if you dig into it. The following grab is taken from a debug session and shows the structure of the error object.

enter image description here

Upvotes: 2

Lefteris S
Lefteris S

Reputation: 1672

Cloud support here, I can see that you also filed a github issue about this where it was made clear that a) this is not related to the node.js client library but to the BigQuery API itself and b) you can at least get the row where the error occurs. In any case, I don't think it's in anyway possible to also get the missing field at this moment, but I filed a feature request that you can follow for updates.

Upvotes: 2

Related Questions