CODEWITHSUNDEEP
google-bigquery
khusnanadia
khusnanadia

Reputation: 853

Is it possible to export all Bigquery scheduled queries listed into a csv?

I need to create programmatic approach in getting the updated Bigquery scheduled queries listed in csv (or Gsheet / Bigquery table if possible). But I cannot find the related documentation for that. For now I can only select all text manually from Bigquery scheduled query page. Below are the information needed:

But with the new scheduled query is still being created, it is getting more complicated to track with the list is still growing.

Upvotes: 0

Views: 553

Answers (1)

Prajna Rai T
Prajna Rai T

Reputation: 1820

To list the scheduled queries with the bq CLI:

bq ls --transfer_config --transfer_location=US --format=prettyjson

For viewing the details of a Schedule Query:

bq show --transfer_config [RESOURCE_NAME]
# [RESOURCE_NAME] is the value from the above bq ls command

In python you can use the below code to list transfer configurations in a project.

from google.cloud import bigquery_datatransfer

transfer_client = bigquery_datatransfer.DataTransferServiceClient()

project_id = "my-project"
parent = transfer_client.common_project_path(project_id)

configs = transfer_client.list_transfer_configs(parent=parent)
print("Got the following configs:")
for config in configs:
    print(f"\tID: {config.name}, Schedule: {config.schedule}")

For more information you can refer to link1 and link2.

Upvotes: 1

Related Questions