Reputation: 77
We have a client requirement to copy data/rows from Oracle Database(Live feed) to SQL Database on daily basis.
Source database tables contains around 11 million rows at average on each table and some tables contain Modified date field which has trigger on it.
How do I create a package to copy all the rows to SQL Table which takes less time?
Currently I have created a package [using modified date field as the main one] using slowly changing dimension transformation but this scan all the data in source table thus taking more time, Is there a way to scan the table fast and load only new/updated records?
Upvotes: 1
Views: 416
Reputation: 55
You should use Merge script in this case which is faster then SCD(Slow and changing Dimention) for heavy data migration. Call a stored procedure from SSIS and Merge the data into destination which is updated or inserted in the source.
Upvotes: 1