CODEWITHSUNDEEP
google-bigquery
kermatt
kermatt

Reputation: 1672

Partitioning based on column data?

When creating a partitioned table using bq mk --time_partitioning_type=DAY are the partitions created based on the load time of the data, not a date key within the table data itself?

To create partitions based on dates within the date, is the current approach to manually create sharded tables, and load them based on date, as in this post from 2012?

Upvotes: 4

Views: 3669

Answers (2)

Timothy Liu
Timothy Liu

Reputation: 31

Good news, BigQuery currently supports 2 type data partition, included partition by column. Please check here.

I like the feature: An individual operation can commit data into up to 2,000 distinct partitions.

Upvotes: 3

Mikhail Berlyant
Mikhail Berlyant

Reputation: 172944

Yes, partitions created based on data load time not based on data itself
You can use partition decorator (mydataset.mytable1$20160810) if you want to load data into specific partition

Per my understanding, partition by column is something that we should expect to be supported at some point - but not now

Upvotes: 3

Related Questions