CODEWITHSUNDEEP
google-bigquery
Felipe Carlo
Felipe Carlo

Reputation: 349

Query error: Resources exceeded during query execution: The query could not be executed in the allotted memory

I'm getting an error when tries to execute the following query: 

select r.* 
from dataset.table1 r
where id NOT IN (select id from staging_data.table1);

It's basically a query to load incremental data on a table. The dataset.table1 has 360k rows and the incremental on staging_data has 40k. But when I try to run this on my script to load on another table, I got the error: 

Resources exceeded during query execution: The query could not be executed in the allotted memory

This started to happen on the last week, before that it was working well.

I looked over for solutions on internet, but all the solutions doesn't work on my case.

Has anyone know how to solve it?

I changed the cronjob time and it worked. Thank you!

Upvotes: 1

Views: 3142

Answers (1)

dsesto
dsesto

Reputation: 8178

You can try using writing the results to another table, as Big Query has a limitation for the maximum response size that can be processed. You can do that either if you are using Legacy or Standard SQL, and you can follow the steps to do it in the documentation.

Upvotes: 3

Related Questions