CODEWITHSUNDEEP
python
Jack_of_All_Trades
Jack_of_All_Trades

Reputation: 11468

Removing duplicates and preserving order when elements inside the list is list itself

I have a following problem while trying to do some nodal analysis:

For example:

my_list=[[1,2,3,1],[2,3,1,2],[3,2,1,3]]

I want to write a function that treats the element_list inside my_list in a following way:

-The number of occurrence of certain element inside the list of my_list is not important and, as long as the unique elements inside the list are same, they are identical.

Find the identical loop based on the above premises and only keep the first one and ignore other identical lists of my_list while preserving the order.

Thus, in above example the function should return just the first list which is [1,2,3,1] because all the lists inside my_list are equal based on above premises.

I wrote a function in python to do this but I think it can be shortened and I am not sure if this is an efficient way to do it. Here is my code:

def _remove_duplicate_loops(duplicate_loop):

        loops=[]
        for i in range(len(duplicate_loop)):

            unique_el_list=[]

            for j in range(len(duplicate_loop[i])):
                if (duplicate_loop[i][j] not in unique_el_list):
                    unique_el_list.append(duplicate_loop[i][j])

            loops.append(unique_el_list[:])

        loops_set=[set(x) for x in loops]
        unique_loop_dict={}

        for k in range(len(loops_set)):
            if (loops_set[k] not in list(unique_loop_dict.values())):
                unique_loop_dict[k]=loops_set[k]

        unique_loop_pos=list(unique_loop_dict.keys())

        unique_loops=[]

        for l in range(len(unique_loop_pos)):
            unique_loops.append(duplicate_loop[l])

        return unique_loops

Upvotes: 0

Views: 89

Answers (2)

Corley Brigman
Corley Brigman

Reputation: 12391

you could do it in a fairly straightforward way using dictionaries. but you'll need to use frozenset instead of set, as sets are mutable and therefore not hashable.

def _remove_duplicate_lists(duplicate_loop):
     dupdict = OrderedDict((frozenset(x), x) for x in reversed(duplicate_loop))
     return reversed(dupdict.values())

should do it. Note the double reversed() because normally the last item is the one that is preserved, where you want the first, and the double reverses accomplish that.

edit: correction, yes, per Steven's answer, it must be an OrderedDict(), or the values returned will not be correct. His version might be slightly faster too..

edit again: You need an ordered dict if the order of the lists is important. Say your list is

[[1,2,3,4], [4,3,2,1], [5,6,7,8]]

The ordered dict version will ALWAYS return

[[1,2,3,4], [5,6,7,8]]

However, the regular dict version may return the above, or may return

[[5,6,7,8], [1,2,3,4]]

If you don't care, a non-ordered dict version may be faster/use less memory.

Upvotes: 2

Steven Rumbalski
Steven Rumbalski

Reputation: 45542

from collections import OrderedDict
my_list = [[1, 2, 3, 1], [2, 3, 1, 2], [3, 2, 1, 3]]

seen_combos = OrderedDict()
for sublist in my_list:
    unique_elements = frozenset(sublist)
    if unique_elements not in seen_combos:
        seen_combos[unique_elements] = sublist
my_list = seen_combos.values()

Upvotes: 4

Related Questions