Kim
Kim

Reputation: 403

Do nothing with (no) match output in the lookup transformation (SSIS)

I'm new to SSIS as well as Stackoverflow. Here's my situation.

I'm building a database and an archive database which need to be synced daily. The records in the database need to be copied to the archieve. I use SSIS and daily jobs to do this. Obviously, I don't want SSIS to load all the data everytime, only the new records (that aren't in the archive yet). I want to use the lookup transformation to achieve this. I'm testing it and it works, it only copies the new data to the "no match output" (which is my archive). But I linked the "match output" to a new destination. But as there are many columns and records, it would be way too much to keep all those redundant data (ofcourse I can purge the data but I don't want to have those extra columns in the first place!). I actually don't want the "match output" to be send to anywhere! How to do this? Or some solution that is more efficient than what I'm doing now (sending the matched outputs to new destinations and deleting those columns or records later on).

P.S. I already found this question on stackoverflow which is a similar question (except the other way around, the TS wants to do nothing with "no match output"): Sending no match output rows to nowhere But the thing is that I don't want to download/use "thrash destination", I'd rather use everything that is already built in SSIS itself. And I don't understand how the derived column transformation could solve the problem. There are no other answers on that question, so that's why I made a new thread.

Can anyone help me with this? (and excuse my English, it's not my native language)

Upvotes: 1

Views: 1916

Answers (1)

Tristan
Tristan

Reputation: 1014

Just don't map the match output. In case that gives an error map it to a row count, that way you can keep track of the amount of duplicate data being handled.

Would be even better to filter this in the source component though, for performance reasons

Upvotes: 2

Related Questions