CODEWITHSUNDEEP
google-bigquery
Graham Polley
Graham Polley

Reputation: 14781

Sharing BigQuery's cache across multiple users

As I understand it, BigQuery's caching mechanism is on a per user basis. But we'd like to be able to share the cache on something like a project/dataset/table level.

For example, John & Mary both work on the same Google project XYZ. They love using BigQuery, and both query the table Bar in dataset Foo i.e. XYZ:Foo.Bar to get beautiful insights from their data.

John logs in and writes a query against XYZ:Foo.Bar which takes 10 seconds to execute. A few minutes later Mary logs in and composes the exact same query on XYZ:Foo.Bar. It also takes 10 seconds, but she does not get a cache hit.

Is there anything that can be done to share the query cache across users i.e. on a project/dataset/table level? Or have I missed something obvious?

Upvotes: 3

Views: 597

Answers (1)

Felipe Hoffa
Felipe Hoffa

Reputation: 59165

BigQuery doesn't share cache across users for privacy reasons - but it could be an interesting feature request to propose: https://code.google.com/p/google-bigquery/.

An alternative you could implement today is a proxy that would connect to BigQuery on behalf of your users with a service account. For example, you get the BigQuery native cache and an application level cache when using http://demo.redash.io. Same with Datalab - as it uses a service account by default, results are cached for users in the same project.

Upvotes: 4

Related Questions