CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquery
Tom Larkworthy
Tom Larkworthy

Reputation: 2354

Is intermediate IO in Bigquery queries charged?

I have a fairly complex Bigquery query and it seems to cost more than I expect. It has 97 intermediate stages... are those charged?

Upvotes: 0

Views: 74

Answers (2)

Steven Ensslen
Steven Ensslen

Reputation: 1376

I agree that the documentation is unclear. I can assure you from watching the query costs for several bigquery projects for years that the intermediate stages are not billed. But I don't have a reference for that.

I can say that you can run a query which takes a day or more of CPU time but which gets billed around $0.01. Don't write those queries, but if you happen to accidentally get your nested CTE's into chaos, it doesn't matter from the billing perspective. It's the initial read of the tables which are billed.

Leaving out the partition key on an otherwise trivial query can be very expensive, even if the slot time is in seconds.

Upvotes: 0

Alessandro
Alessandro

Reputation: 655

You can get the value of how much data will be scanned (and therefore charged) by your query using the --dry-run switch from the CLI or by looking at the right end of the UI section where you run and set up your query.

BigQuery pricing model is per byte read. For my understanding, at the moment, if you reference a table in multiple CTE you will get charged one. But this might depend on how the query is written.

The best practice is always using the --dry-run feature which is very accurate.

Upvotes: 1

Related Questions