Reputation: 1
I have a file with 50,000 lines of data in 3 columns- Unique ID, Start Date, and End Date.
Using Power Pivot, I need to determine if any records with the same Unique ID have any overlapping dates. Each Unique ID appears about 5 times.
In excel, I would use a formula
SUMPRODUCT: =SUMPRODUCT(($B3<=$C$3:$C$13)*($C3>=$B$3:$B$13)*($A$3:$A$13=A3))>1
While this formula works really well in excel, with 50k+ records, this breaks my computer.
I was wondering, how would I perform that same calculation in power pivot/query.
Example of the data and calculation.
Thank you so much!
Upvotes: 0
Views: 132
Reputation: 953
following a PowerQuery M-Code, this will solve your problem. Don't know how long it will take for 50k rows:
let
Quelle = Excel.CurrentWorkbook(){[Name="tab_Dates"]}[Content],
Change_Type = Table.TransformColumnTypes(Quelle,{{"Unique ID", type text}, {"Start Date", type date}, {"End Date", type date}}),
add_List_Dates = Table.AddColumn(Change_Type, "List_Dates", each List.Dates([Start Date], Duration.Days([End Date]-[Start Date])+1 , #duration(1,0,0,0))),
expand_List_Dates = Table.ExpandListColumn(add_List_Dates, "List_Dates"),
add_CountIF_ID_Date = Table.AddColumn(expand_List_Dates, "CountIF_ID_Date", (CountRows) =>
Table.RowCount(
Table.SelectRows(
expand_List_Dates,
each
([Unique ID] = CountRows[Unique ID] and [List_Dates] = CountRows[List_Dates])))),
Change_Type_2 = Table.TransformColumnTypes(add_CountIF_ID_Date,{{"CountIF_ID_Date", type text}}),
ChangeValue_CountIF_ID_Date = Table.ReplaceValue(Change_Type_2, each [CountIF_ID_Date], each if [CountIF_ID_Date] <> "1" then "FALSE" else "TRUE",Replacer.ReplaceText,{"CountIF_ID_Date"}),
Remove_Column_List_Dates = Table.RemoveColumns(ChangeValue_CountIF_ID_Date,{"List_Dates"}),
Remove_Duplicates = Table.Distinct(Remove_Column_List_Dates)
in
Remove_Duplicates
Upvotes: 0