CODEWITHSUNDEEP
databricksdelta-lakescddelta-live-tables
Rajib Deb
Rajib Deb

Reputation: 1774

SCD-2 using delta live table

Delta live table now has the capability to do SCD Type 2 changes. But after going through this feature, I understood that this will work if I have only one new row with a new effective date. In the scenario where I have two new rows with two different effective dates, how do we handle such scenarios. For example, if I have a discount that is effective from today till next 3 months and one more discount which is effective after three months. In this case, whenever a change record comes, i will have to sort the rows to get the latest effective data and change that and not the future one

Thanks

Upvotes: 2

Views: 424

Answers (2)

Mojgan Mazouchi
Mojgan Mazouchi

Reputation: 355

In a scenario like this, you can make a struct out of two effective dates in your data and pass that struct to the sequence by to handle the ordering for you.

Upvotes: 1

BobGally
BobGally

Reputation: 11

Does those two discount have any unique key? If that is the case you can build the type2 using that. If not you can make the discount and discount end date as a key.

Upvotes: 0

Related Questions