CODEWITHSUNDEEP
sqlazure-data-factory
user16402180
user16402180

Reputation: 21

Duplicates getting inserted in data flow transformation - ADF pipeline

I am new to ADF. I have a sql query that I need to convert to ADF pipeline. I have table1 in linkedserver A and table2 in Linkedserver B. I need to import data from Table1 to table2. I have below sql query

select A.Id from linkedserver1.databasename.dbo.table1 A left join  linkedserver2.databasename.dbo.table2 B on a.id = b.id
where b.id is null

I need to convert this to pipeline I don't want to import record that is already existing in table B. below pipline is inserting duplicate records. I tried using left join . enter image description here

Upvotes: 0

Views: 643

Answers (1)

ray
ray

Reputation: 15266

You just need 1 more filter activity after your left join and before sink

You can use isNull(...) in the Filter on expression. enter image description here

Upvotes: 0

Related Questions