CODEWITHSUNDEEP
gogoogle-bigquery
Gilo
Gilo

Reputation: 720

Slow uploads to bigQuery using golang client

I am using deprecated BigQuery Golang library. Lately uploads became much slower. I wonder if updating the client to a new library will make a performance improvement. I couldn't find in the documentation anything saying about performance improvement.

Old deprecated library I am currently using: https://github.com/googleapis/google-api-go-client/tree/main/bigquery/v2

New client: https://github.com/googleapis/google-cloud-go/tree/main/bigquery

Should I expect to get performance improvement with the new library?

Upvotes: 0

Views: 302

Answers (1)

shollyman
shollyman

Reputation: 4384

The cloud.google.com/go/bigquery package uses the google-api-go-client bigquery package under the hood. I wouldn't expect an improvement in raw network throughput from the switch, but you might see improvements due to how the cloud package does more of the low level API management for you.

To really answer the question, you'd need to be able to detail what you're doing and try to quantify the details a bit more: are you sending bytes to GCS, and then constructing a load job? Sending an inline upload as a load job? Streaming data?

Upvotes: 1

Related Questions