CODEWITHSUNDEEP
google-bigquery
Dragon
Dragon

Reputation: 453

How can we load files under folder into Big Query from Google Storage?

I have a folder of files and subdirectories in the google storage. Want to upload all the files under this folder into bqdataset.bqdatatable using a schema.

If I specify a single file it successfully uploads the file but while giving a folder path it is unable to do that.

Please advise.

Regards.

Upvotes: 0

Views: 2971

Answers (1)

Mikhail Berlyant
Mikhail Berlyant

Reputation: 173028

It must work! You should use below as a source

gs://your_bucket/your_folder/*

of course assuming all files have the same schema (which you should also provide) and you load your folder into one table

it works both using BigQuery UI or programmatically with API https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load

Upvotes: 2

Related Questions