CODEWITHSUNDEEP
google-bigquery
Bobbylank
Bobbylank

Reputation: 1946

Removing BigQuery Public Dataset

Does anyone know of any way to remove the public datasets from a BigQuery project?

Though the risk is very low, I don't want my users to be able to run queries against them and rack up costs.

Thanks

Upvotes: 4

Views: 2068

Answers (3)

Sujit Dhamale
Sujit Dhamale

Reputation: 1446

if you closed current tab , public data set will disappear from google BigQuery page

Upvotes: 2

Hugh
Hugh

Reputation: 1361

Its an old question, but for those who just want to unpin the "bigquery-public-data" to tidy up the resources list, you can click the name on the side, then on the far right of the info pane there is an "unpin project button". Click that.

Upvotes: 1

MonicaPC
MonicaPC

Reputation: 374

The whole point of public datasets is that everyone has access to them so they can test BigQuery. Even if a feature request will create the option to disable the listing in the panel of the BigQuery web UI, the users will still have access and could query the public datasets.

It will be more practical to use custom quotas.

So you would create a project with a number of users that share a quota that you consider enough for their activities. When the established quota is reached BigQuery stops and the users receive an error message when trying to run queries.

Another useful tool is creating budget alerts with a desired level that you can set taking into account the previous month's spend. The alert will notify you when the project's bill have reached the amount you set and can save you from bad surprises.

In addition, implementing the Audit Logs in your project will give comprehensive overview on the BigQuery operations. Check this example of an Audit Logs query that will give details on the performed queries. Of course, you will find out about the use of a public dataset after it happens but this will point out who’s the user that performed the query and you can reinforce the administration policy of not inquiring public datasets. To get information on the performed query, including the interrogated dataset, use this field when querying the Audit Logs:

'protopayload_auditlog.servicedata_v1_bigquery.jobCompletedEvent.job.jobConfiguration.query.query'

As a last resort, you can create a designated project for your users to query the public datasets and to make sure it will not create additional costs, you can remove the billing account. Though, by doing so you can only query 1 TB of data per month, the BigQuery always free usage tier.

Also keep in mind about this best practices to limit the queries costs.

Upvotes: 0

Related Questions