CODEWITHSUNDEEP
google-bigquery
Sourygna
Sourygna

Reputation: 719

BigQuery streaming and partitions: when is _PARTITIONTIME really evaluated?

_PARTITIONTIME represents the time (truncated to the day) when a row is inserted into BigQuery.

However, when looking closely at the streaming mechanism (https://cloud.google.com/blog/products/gcp/life-of-a-bigquery-streaming-insert ), we can see 3 different "insertion times" when a row is inserted into BigQuery:

Does somebody knows which one of those 3 moments correspond to _PARTITIONTIME ?

Upvotes: 1

Views: 396

Answers (1)

Hua Zhang
Hua Zhang

Reputation: 1551

When the row is still in the streaming buffer, _PARTITIONTIME is null for this row; after the row is extracted, the extraction time is the _PARTITIONTIME for this row. An exception is that when the row is streamed into a partition directly, "table$20180101". In this case the _PARTITIONTIME is always "2018-01-01".

Upvotes: 4

Related Questions