CODEWITHSUNDEEP
azureazure-data-factory
BlakeB9
BlakeB9

Reputation: 647

Seeing more records in some tables found in sink when using Data Factory (Upsert) to copy tables from one database to another

I've got a pipeline in my Data Factory that is copying data over from a couple of schemas from one database to another. I am also using Upsert and defining key columns. This has ran a few times- for some reason I'm seeing more records in some of my sink database tables compared to the source tables. The other tables that aren't like this seem to be spot on. What is causing the discrepancy?

Upvotes: 0

Views: 88

Answers (1)

Nandan
Nandan

Reputation: 4945

My guess is there are some records in the source which were synced earlier in sink but eventually some records were deleted from source and upsert logic in copy activity cannot handle that scenario (of even deleting the record from sink)

Hope this clarifies.

Upvotes: 1

Related Questions