CODEWITHSUNDEEP
google-bigquery
kazunori_279
kazunori_279

Reputation: 3

How many concurrent uploads from Google Cloud Storage to BigQuery we can request?

On the doc, it states that you can upload multiple files from Storage to a table in separate requests:

Create a CSV file to populate your table - Data used to populate your table is uploaded as a CSV file. Maximum size for a single source file is 4GB, but you can import multiple source files to a table, either in a single request or in separate requests.

Can those requests be issued concurrently? If yes, what will be a maximum concurrent number of uploading requests per account/bucket/table? If there's no explicit quota on those numbers, does it mean you can increase uploading throughput as much as you want by issuing the uploading requests?

Thanks,

Kaz

Upvotes: 0

Views: 1038

Answers (2)

Felipe Hoffa
Felipe Hoffa

Reputation: 59325

2013 update:

In summary, each import request can have a maximum of 10,000 files (each has to be under 1Tb, without new lines in the strings).

https://developers.google.com/bigquery/docs/quota-policy#import

Upvotes: 0

Michael Manoochehri
Michael Manoochehri

Reputation: 7887

The BigQuery quotas are listed here: https://developers.google.com/bigquery/docs/quota-policy

In summary, each import request can have a maximum of 500 files (each has to be under 4Gb). You can make two import requests per minute, with a max of 1,000 import requests per day

Upvotes: 2

Related Questions