CODEWITHSUNDEEP
google-app-enginegogoogle-bigquerygoogle-cloud-storage
Zanndorin
Zanndorin

Reputation: 370

Extracting result from Google BigQuery to cloud storage golang

I am using the following GoLang package: https://godoc.org/cloud.google.com/go/bigquery

My app runs in Google App Engine

If I have understood the documentation correctly it should be possible to extract the result of a job/query to Google Cloud Storage using a job. I don't think the documentation is very clear and was wondering if anyone has an example code or other help.

TL:DR

  1. How do I get access to the temporary table when using Go Lang instead of command line.

  2. How do I extract the result of my Bigquery to GCS

** EDIT **

Solution i used

I created a temporary table and set it as the Dst (Destination) for the Query result and created an export job with it.

dataset_result.Table(table_name).Create(ctx, bigquery.TableExpiration(time.Now().Add(1*time.Hour)))

Update 2018:

https://github.com/GoogleCloudPlatform/google-cloud-go/issues/547

To get the table name:

q := client.Query(...)
job, err := q.Run(ctx)
// handle err
config, err := job.Config()
// handle err
tempTable := config.(*QueryConfig).Dst

Upvotes: 0

Views: 2148

Answers (1)

Graham Polley
Graham Polley

Reputation: 14781

How do I extract the result of my BigQuery to GCS

You cannot directly write the results of a query to GCS. You first need to run the query, save the results to a permanent table, and then kick off an export job to GCS.

https://cloud.google.com/bigquery/docs/exporting-data

How do I get access to the temporary table when using Go Lang instead of command line.

You call use the jobs API, or look in the query history if using the web UI. See here and here.

https://cloud.google.com/bigquery/querying-data#temporary_and_permanent_tables

Upvotes: 1

Related Questions