CODEWITHSUNDEEP
pythonpython-3.x
bli00
bli00

Reputation: 2787

Remove duplicate list from list

First of all let me acknowledge that this question has been asked before, but the answers either seem outdated or unsatisfactory. The question is given a list of unsorted list, how can we remove duplicates in the most efficient and elegant manner? (i.e. using the shortest syntax vs. the fastest computational time)

Example:

Given [[1,2,3],[],[2,-2],[3,2,1]], we want [[1,2,3],[],[2,-2]]. Note that whether [1,2,3] or [3,2,1] doesn't matter.

Upvotes: 2

Views: 110

Answers (1)

dawg
dawg

Reputation: 103744

You can do:

>>> li=[[1,2,3],[],[2,-2],[3,2,1]]
>>> {frozenset(e) for e in li}
{frozenset({1, 2, 3}), frozenset({2, -2}), frozenset()}
>>> [list(x) for x in {frozenset(e) for e in li}]
[[1, 2, 3], [2, -2], []]

The key is to use frozenset since a set is not hashable. Note the order may change with this method.

If you want to maintain the same order, you can do:

>>> seen=set()
>>> [e for e in li if frozenset(e) not in seen and not seen.add(frozenset(e))]
[[1, 2, 3], [], [2, -2]]

If there is a possibility of repeated elements within the sublists, you can sort the sublists and use a representation of that:

li=[[1,2,3],[],[2,-2],[3,2,1],[1,1,2,2,3],[1,2,1,2,3]]
seen=set()
nli=[]
for e in li:
    re=repr(sorted(e))
    if re not in seen:
        seen.add(re)
        nli.append(e)

>>> nli
[[1, 2, 3], [], [2, -2], [1, 1, 2, 2, 3]]

(Note: You can use tuple instead or repr if desired. Either produces a hashable immutable result)

Upvotes: 5

Related Questions