CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquery
hanlei
hanlei

Reputation: 3

how to download the big result in bigquery

I am new to gcp. My mission is to download the query result in patents dataset, but the result is too large. I cannot download it directly because gcp only supports download 16000 lines data.

I select several columns and the data is already too large

SELECT country_code, kind_code, application_kind, family_id, publication_date, filing_date, cpc.code as cpc_code, ipc.code as ipc_code
FROM
`patents-public-data.patents.publications` p
cross join unnest(p.cpc) as cpc
cross join unnest(p.ipc) as ipc

I expect I can download the result table, or download by the country_code in different tables.

Upvotes: 0

Views: 835

Answers (2)

guillaume blaquiere
guillaume blaquiere

Reputation: 75910

To complement the response of @Christopher, and for achieving your download, here the steps to perform:

  • Perform your query
  • Save result in (temporary) table
  • Extract table to Google storage bucket
  • Download file(s) where you want, manually in console or with gsutil tool

Note that there is no limitation on size, but you can have more than 1 file of the result is huge. Take care of format for nested field and prefer gzip compression for faster download!

Upvotes: 1

Christopher
Christopher

Reputation: 941

You can write the result in another table or export table data in Cloud Storage (take note of the export limitations)

Upvotes: 0

Related Questions