CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquery
Dragon
Dragon

Reputation: 453

How to load .zip files into BigQuery?

We are able to load uncompressed CSV files and gzipped files completely fine. However, if we want to load CSV files compressed in ".zip" - what is the best approach to move ahead? Will we need to manually convert the zip to gz or BigQuery has added some support to handle this?

Thanks

Upvotes: 7

Views: 10166

Answers (2)

user15338850
user15338850

Reputation: 11

You can try 42Layers.io for this. We use it to import ziped CSV files directly from FTP into BQ, and then set it on a schedule to do it every day. They also let you do field mapping to your existing tables within BQ. Pretty neat.

Upvotes: 1

Mikhail Berlyant
Mikhail Berlyant

Reputation: 173161

BigQuery supports loading gzip files

The limitation is - If you use gzip compression BigQuery cannot read the data in parallel. Loading compressed CSV data into BigQuery is slower than loading uncompressed data.

Upvotes: 4

Related Questions