CODEWITHSUNDEEP
google-bigquery
Felipe Hoffa
Felipe Hoffa

Reputation: 59165

Querying more than a 1000 tables per query?

There is a 1000 table limit on queries. However, some of our datasets have over 1000 tables.

Any plans on increasing the allowed tables per query limit?

Upvotes: 2

Views: 605

Answers (1)

Felipe Hoffa
Felipe Hoffa

Reputation: 59165

While querying the union of a series of tables is a good pattern in BigQuery, there is a cost for this: Each additional table involved in a query makes said query marginally slower.

That's why there is a 1000 tables limit: Approaching this scale, the query gets noticeable slower, and alternative methods become more attractive.

Can you pre-combine said >1000 tables into a big one for queries that would require going through a lot of tables otherwise?

Upvotes: 1

Related Questions