CODEWITHSUNDEEP
databasegoogle-cloud-platformgoogle-cloud-firestoregoogle-bigquery
Alexander Richter
Alexander Richter

Reputation: 170

Use Firestore to store critical values in Big Query

I need Big Query to efficiently analyse TB large datasets.

Every couple of seconds I need to check the maximum and minimum value from some column in the big query table. Even with an appropriate clustering this has become quite expensive.

Would it make sense to store these values in a separate firestore database? Any thoughts? Thanks

Upvotes: 0

Views: 50

Answers (1)

Matthew Hynes
Matthew Hynes

Reputation: 46

Presumably you know the min and max values from the previous time you checked, so you should only need to check the min and max values from the new data that has arrived since the last check. You don't need to query the whole table every time.

Upvotes: 2

Related Questions