CODEWITHSUNDEEP
google-bigquery
BK25
BK25

Reputation: 1

Bigquery data transfer duration from intraday table to daily table

I am using firebase analytics and bigguery with average of 50~60 GB daily data.

For the most recent daily table, a query gives different result from yesterday even if query conditions are exact same including target date.

I just found that there are 1~2days gap between table creation date and last modified date.

I assume the difference between the query results are because of this. (Calculating on different data volume, maybe)

Is this date gap means a single daily table needs at least 2 days to be fully loaded from intraday table?

Thanks in advance.

biqguery table info

Upvotes: 0

Views: 1214

Answers (1)

rmesteves
rmesteves

Reputation: 4085

In the documentation we can find the following information:

After you link a project to BigQuery, the first daily export of events creates a corresponding dataset in the associated BigQuery project. Then, each day, raw event data for each linked app populates a new daily table in the associated dataset, and raw event data is streamed into a separate intraday BigQuery table in real-time.

It seems that the intraday table is loaded to the main table each day and if you want to access this data in real-time you`ll have to use this intraday separate table.

If this information doesn`t help you, please provide some extra information so I can help you more efficiently.

Upvotes: 1

Related Questions