CODEWITHSUNDEEP
google-bigquerygoogle-cloud-platform
Daryl Wenman-Bateson
Daryl Wenman-Bateson

Reputation: 3954

Service throttling in BigQuery

How can I tell if throttling is occurring within Google BigQuery?

I have several BigQuery jobs lifting Google Analytics data which typically complete within 10-15 seconds. Different jobs have intermittently started to slow down to 2-3 minutes, before returning to their normal speed in subsequent runs.

The jobs are completing successfully. I can't see any pattern in the slow down - its not the same job, nor at the same time and there has not been a surge in traffic volume.

Could it be service throttling based on the Google BigQuery project? Is there any way to monitor this in the Cloud Console?

Is there anything else other than service throttling that I should be considering?

Upvotes: 0

Views: 774

Answers (1)

Graham Polley
Graham Polley

Reputation: 14791

You're not being throttled. Instead, you're using a multi-tenant system, so you need to share all the resources with your neighbours. Those slower jobs are most likely occurring when there is more demand on the cluster. Or maybe it's just because there's more data to load.

Under the hood, load jobs are just normal federated queries, except you don't pay for them. And just like queries, they too are constrained by the demand on the resources/cluster at any given time.

Upvotes: 1

Related Questions