CODEWITHSUNDEEP
sqlsnowflake-cloud-data-platformetlauto-incrementdbt
Idan Kogan
Idan Kogan

Reputation: 1

dbt Incremental Model Issue with Snowflake Autoincrement Column

I am encountering an error while utilizing an autoincrement column named col_2 in Snowflake within my dbt incremental model. The column col_2 is defined in Snowflake as:

col_2(18,0) autoincrement start 1 increment 1 order.

When executing my incremental model, I receive the following error message:

invalid identifier 'col_2'.

My model is structured as follows:

{{
    config(
        materialized='incremental',
        unique_key='col1',
    )
}}

SELECT DISTINCT col1
FROM {{ ref('other_model') }}

It appears that DBT converts my model into something akin to:

...

when not matched then insert
    ("col1", "col2")
values
    ("col1", "col2")

However, col_2 is an autoincrement column in Snowflake, and I do not wish to insert values into it manually as Snowflake handles this automatically.

Could you please advise on how to resolve this issue?

I have attempted various methods to address this issue, but I am uncertain if I am heading in the correct direction. Here are the modifications in the model I've tried

  1. on_schema_change: ignore
  2. merge_exclude_columns = ['col_2']

I am seeking a solution to ignore specific columns in the insertion query.

Thank you.

Upvotes: 0

Views: 558

Answers (0)

Related Questions