CODEWITHSUNDEEP
google-bigquerybq
user9694017
user9694017

Reputation:

BigQuery CLI: load commands stays pending

I have a csv file on my computer. I would like to load this CSV file into a BigQuery table. I'm using the following command from a terminal: 

bq load --apilog=./logs --field_delimiter=$(printf ';') --skip_leading_rows=1 --autodetect dataset1.table1 mycsvfile.csv myschema.json

The command in my terminal doesn't give any output. In the GCP interface, I see no job being created, which makes me think the request doesn't even reach GCP. In the logs file (from the --apilog parameter) I get informations about the request being made, and it ends with this: 

INFO:googleapiclient.discovery:URL being requested: POST https://bigquery.googleapis.com/upload/bigquery/v2/projects/myproject/jobs?uploadType=resumable&alt=json

and that's it. No matter how long I wait, nothing happens.

Upvotes: 0

Views: 142

Answers (2)

rsantiago
rsantiago

Reputation: 2099

You are mixing --autodetect with myschema.json, something like the following shoud work:

bq load --apilog=logs \
--source_format=CSV \
--field_delimiter=';' \
--skip_leading_rows=1 \
--autodetect \
dataset.table \
mycsvfile.csv

If you continue having issues, please post the content of the apilog, the line you shared doesn't seem to be an error. There should be more than one line and normally contains the error in a json structure, for instance:

"reason": "invalid",
      "message": "Provided Schema does not match Table project:dataset.table. Field users is missing in new schema"

Upvotes: 1

Enrique Zetina
Enrique Zetina

Reputation: 835

I'm not sure why you are using

--apilog=./logs

I did not find this in the bq load documentation, please clarify.

Based on that, maybe the bq load command could be the issue, you can try with something like:

bq load \
--autodetect  \
--source_format=CSV \
--skip_leading_rows= 1  \
--field_delimiter=';'
dataset1.table1 \
gs://mybucket/mycsvfile.csv \
./myschema.json

If it fails, please check your job list to get the job created, then use bq show to view the information about that job, there you should find an error messag which can help you to determine the cause of the issue.

Upvotes: 0

Related Questions