CODEWITHSUNDEEP
google-bigquery
Bhavik Desai
Bhavik Desai

Reputation: 35

Google BigQuery How to Load custom columns from csv?

I have a doubleclick csv file with 20 columns (Timestamp,AdvertiserId,ActionName,Brower,OSID ...) without any header. I would like to ingest only first 3 columns into a BQ table. Is there any way to achieve that without mapping each and every column one-by-one manually into BQ's UI (create new_table ->"Schema" section)? Fields in CSV is comma separated and newlines are defined as semi-colon';'.

Upvotes: 1

Views: 1631

Answers (1)

Victor Mota
Victor Mota

Reputation: 1269

There are two possible ways to do that: BigQuery: Load from CSV, skip columns

In your case I would probably suggest the second approach. Set the ignoreUnknownValues flag and pass in a schema with just the first three columns. For example:

bq load --ignore_unknown_values dataset.new_table gs://path/to/file.csv ~/path/to/schema.json

Upvotes: 1

Related Questions