CODEWITHSUNDEEP
google-bigquerygoogle-cloud-storage
Jeugasce
Jeugasce

Reputation: 197

upload multiple csv from google cloud to bigquery

I need to upload multiple CSV files from my google bucket. Tried pointing to the bucket when creating the dataset, but i received an error. also tried

gsutil load <projectID:dataset.table> gs://mybucket

it didn't work.

I need to upload multiple files at a time as my total data is 2-3 TB and there is a large number of files

Upvotes: 4

Views: 3094

Answers (1)

Brandon Yarbrough
Brandon Yarbrough

Reputation: 38369

You're close. Google Cloud Storage uses gsutil, but BigQuery's command-line utility is "bq". The command you're looking for is bq load <table> gs://mybucket/file.csv.

bq's documentation is over here: https://developers.google.com/bigquery/bq-command-line-tool

Upvotes: 4

Related Questions