CODEWITHSUNDEEP
google-cloud-platformgoogle-bigqueryextractpartitioningbq
emily.mi
emily.mi

Reputation: 159

cannot export partition by date table from BigQuery to GCP

I'm trying to export the partition by day table to the GCP bucket. My main goal is to create daily files in the GCP.

Here is my simple first test for the extraction:

bq extract --destination_format=CSV test.test_table$20210719 gs://test_partition/part_col=20210716/test_*.csv

test_table is a partition table and I do have 7k rows for the date 2021-07-19. However, when I run this line, I get this

BigQuery error in extract operation: Error processing job 'test-net:bqjob_1234456789: Not found: Table test-
net:temp.test_partitiontime0210719 was not found in location EU

As you can see above the table name is temp.test_partitiontime0210719 there is no 2 at the beginning of it. Whenever $ sign is included in the table name, the first character after it is being removed.

I've tried to apply this GCP documentation. Also, I have tried this comment, but it hasn't worked for me.

How can I extract the partition table to the GCP for a specific date?

Upvotes: 0

Views: 176

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75715

It's a linux side effect. If you put $something linux search for an environment variable with the key something.

To skip that, you can use simple quote ' to say to linux: "Hey, do not evaluate this environment variable token", so do this

bq extract --destination_format=CSV 'test.test_table$20210719' gs://test_partition/part_col=20210716/test_*.csv

Upvotes: 1

Related Questions