CODEWITHSUNDEEP
google-cloud-platformgoogle-bigqueryquota
LQuaD
LQuaD

Reputation: 11

Google BigQuery ExtractBytesPerDay usage quota exceeded - What is ExtractBytesPerDay?

I am running a sequence of queries in BigQuery that take data from a table, enrich/transform them and load into other tables within the same project.

Very high level query structure looks as such:

WITH query_1 as (
SELECT columns from Table_A
WHERE some_condition = some_value
),
query_2 as (
SELECT processing_function(columns)
from A)
SELECT * from query_2

I am calling this query through Python and specifying the destination table in the query job config.

The Table_A mentioned above has about 2 TB of data for a day and I am looping this operation for 10 days. After processing just 1 day of data, BigQuery gave the following error

403 Quota exceeded: Your usage exceeded quota for ExtractBytesPerDay. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors

I checked the quota and it is pushing the 10TB quota limit I have but I'm not able to figure out what ExtractBytesPerDay quota really is. I can increase the quota limit, but I would like to evaluate how much I would need to increase it.

So any direction on which operations account for this quota would be helpful. Anybody know what ExtractBytesPerDay quota means??

Upvotes: 1

Views: 2668

Answers (1)

rsantiago
rsantiago

Reputation: 2099

You quota seems to be related to Exporting Data from BQ that extract data to a destination like Cloud Storage.

You should know that there is a new API called BigQuery Storage API that can help to mitigate your quota issue.

In case that don't help you, you might want to contact the Google Cloud Platform Support to get more insights and know how to resolve your error.

Upvotes: 0

Related Questions