CODEWITHSUNDEEP
talendtalend-mdm
ilovebreakfast
ilovebreakfast

Reputation: 1

How to perform conditional de-duplication in talend

I have a table with the employee ID, name, and last effective date. I want to keep only the employee ID with max(last effective date) and discard the other duplicate employee id rows that have an earlier last effective date.

I am trying to implement this usingtAggregate and tFilterRow. I attemped to perform count using tAggregate but it does not capture max(lasteffectivedate).

Upvotes: 0

Views: 80

Answers (1)

Théo Capdet
Théo Capdet

Reputation: 1062

With a tAggreggateRow you can do :

enter image description here

If you want to get the ID too you need to reuse your primary flow in tMap (Main branch) and put the flow with the tAggregateRow in the lookup of the tMap. After that you do a join on name AND date.

Upvotes: 1

Related Questions