CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquery
user14676730
user14676730

Reputation:

GCP BigQuery shows "This script will process 2TB when run" for a simple query with "limit 10"

When I select my query for a large partition table in GCP BigQuery,

select * from <myBQtable> limit 10;

I haven't clicked "Run" button yet and it told me "This script will process 2TB when run."

I wonder why a "limit 10" query will still have this big cost.

Upvotes: 0

Views: 1362

Answers (2)

guillaume blaquiere
guillaume blaquiere

Reputation: 75715

With BigQuery, you pay for the volume of data that you scan, not that you get. You have different way to optimize the volume of data that you query with partitioning and clustering.

But also, by avoiding to get ALL the column (select *), but by selecting only the relevant column (BigQuery has a column oriented storage and if you don't mention a column you don't load/scan it). You have guidelines to save cost in BigQuery

Upvotes: 3

Sakshi Gatyan
Sakshi Gatyan

Reputation: 2116

LIMIT is not used to control costs in BigQuery. It only reduces the total time taken to display the results.

So for SELECT * FROM [table] LIMIT 10, the query will read the whole table but display only 10 rows.

Upvotes: 2

Related Questions