CODEWITHSUNDEEP
azureazure-synapseazure-data-factory
Waris Souran
Waris Souran

Reputation: 21

Move Data from Oracle Server to Azure Blob Storage with Incremental Changes or delta load

I have a problem with the incremental load I want to create an delta load (incremental load) pipeline from the Oracle on-premise server to Azure data lake(blob storage) I don't have Azure SQL. I just want to push in blob storage as a CSV file. in my case, I am confused about how can I do this?. someone told me in your case you have to use parquet delta. please help me with this I am stuck for many days.

First I loaded with full load (copy activity) now I want load only new data from oracle server with triger every week on sunday.

PLease help in this issue! Thanks in advance Waris

I need help in pipeline to build increment load only new rows from oracle server to azure data lake

Upvotes: 0

Views: 656

Answers (1)

user21374523
user21374523

Reputation:

In the copy activity, instead of selecting the table option in the source, you can select Query and there you can write your select statement with where clause on any timestamp column (ex: loadtimestamp >= getdate()-7) for incremental data load.

copy activity

Upvotes: 0

Related Questions