CODEWITHSUNDEEP
node.jsgoogle-cloud-platformgoogle-bigquery
Adamo Figueroa
Adamo Figueroa

Reputation: 446

problem when restarting tables and insert data in bigquery using node api

I have unexpected behaviour when loading data into BigQuery just after creating the schema.
I'm using Node API to insert data with BigQuery streaming API.
In order to reset the data I delete and create the tables before loading any data.

My Problem: the first time it works fine, but if I execute it again it fails.

The process always delete and creates the table schema, but does not insert the data until I wait a moment to execute it again.

This is the code which reproduces the case:

async function loadDataIntoBigquery() {
const {BigQuery} = require('@google-cloud/bigquery')
const tableName = "users"
const dataset = "data_analisis"
const schemaUsers = "name:string,date:string,type:string"
const userData = [{name: "John", date: "20/08/93", type: "reader"}, {
    name: "Marie",
    date: "20/08/90",
    type: "owner"
}]

try {
    const bigquery = new BigQuery()
    await bigquery.createDataset(dataset).then(err => console.log("dataset created successfully")).catch(err => {
        console.log("warn: maybe the dataset already exists")
    })
    await bigquery.dataset(dataset).table(tableName).delete().then(err => console.log("table deleted successfully")).catch((err) => {
        console.log("Error: maybe the table does not exist")
    })
    await bigquery.dataset(dataset).createTable(tableName, {schema: schemaUsers}).then(() => console.log("table created successfully")).catch(err => console.log("Error: maybe the table already exists"))
    await bigquery.dataset(dataset).table(tableName).insert(userData).then((data) => console.log("Ok inserted ", data)).catch(err => console.log("Error: can't insert "))
} catch (err) {
    console.log("err", err)
}

}

to verify that the data was inserted I'm using this query

select * from `data_analisis.users`

Upvotes: 4

Views: 777

Answers (1)

sylvain Gavoille
sylvain Gavoille

Reputation: 1

I have the same issue. As a workaround, i insert data with a query instead :

const query = "INSERT INTO `"+dataset+"."+tableName"` (name, date, type ) VALUES  ("+name+",'"+date+"','"+type+"')";
await bigQuery.query({
    query: query,
    useLegacySql: false,
    location: 'EU'
}, (err) => {
    console.log("Insertion error : ",err);
})

Upvotes: 0

Related Questions