CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquery
ghostcoder
ghostcoder

Reputation: 493

Number of concurrent Big Query load jobs

I am firing thousands of load jobs concurrently to different BigQuery tables. I see some of them to be executed instantly while others are queued. I was wondering how many load jobs can be run concurrently and if there is a way to run more of them instantaneously.

Upvotes: 1

Views: 2117

Answers (1)

TasosZG
TasosZG

Reputation: 1294

As seen in the documentation, the limit is 100 concurrent queries and to raise it you need to contact support or sales:

The following limits apply to query jobs created automatically by running interactive queries and to jobs submitted programmatically using jobs.query and query-type jobs.insert method calls.

Concurrent rate limit for on-demand, interactive queries — 100 concurrent queries

Queries with results that are returned from the query cache, and dry run queries do not count against this limit. You can specify a dry run query using the --dry_run flag or by setting the dryRun property in a query job.

This limit is applied at the project level. To raise the limit, contact support or contact sales.

Upvotes: 3

Related Questions