Coder1990
Coder1990

Reputation: 101

Incremental load from Azure to Snowflake using Matillion

I want to perform incremental load from Azure Blob storage to Snowflake table using Matillion ETL. There is a JDBC incremental load component but it works for a select few databases and Snowflake isn't one of them. I tried searching the documentation but wasn't able to find anything. Any suggestions ?

EDIT: Under properties of my Table Update component

enter image description here

Upvotes: 1

Views: 622

Answers (1)

peterb
peterb

Reputation: 727

The Blob Storage load component does not support update or upsert. To accomplish this, use the Azure Blob storage load component to load to a staging table. Then run a transformation job that reads from the staging table, and uses either the 'Table Output' component with the append option (for Insert), or the 'Table Update' component with the 'include when not matched' option (for Upsert)

enter image description here

Upvotes: 1

Related Questions