CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquerydbtdataform
bha159
bha159

Reputation: 231

Creating tables in BigQuery from Dataform

I am not able to create tables in BigQuery using GCP Dataform. All the tables that I am creating are showing as temporary tables and hence not visible in BigQuery. Below is my source to fetch data from a table of BigQuery.

config {
 type: "declaration",
 database: "xx-prd",
 schema: "mixpanel",
 name: "table1",
 description: "Source test."
}

And this is how I am creating a table

config {
    type: "table"
}

select * 
FROM ${ref('table1')}
where DATE(created_at) = "2023-05-10"
limit 10

When I compile the above code, I can see in the job output in both Dataform and BigQuery UI that a temporary table is created. I tried including schema, bigquery and database in config of above definition with table, hoping that it will change something however nothing changed. I have used type as table, view and incremental, every-time, a temporary table is created.

So I wanted to know how to use an existing BigQuery table as source and create a table in any particular schema of BigQuery from it.

Upvotes: 1

Views: 1275

Answers (1)

BigSmoke
BigSmoke

Reputation: 433

Are you aware that pressing RUN in Dataform will only create a temporary table; to create the table in the actual dataset that you configured, you can use START EXECUTION, for example.

Upvotes: 0

Related Questions