CODEWITHSUNDEEP
pythonlistduplicates
J Cena
J Cena

Reputation: 1053

How to remove duplicate tuples from a list in python?

I have a list that contains list of tuples as follows.

mylist = [['xxx', 879], ['yyy', 315], ['xxx', 879], ['zzz', 171], ['yyy', 315]]

I want to remove the duplicate tuples from mylist and get an output as follows.

mylist = [['xxx', 879], ['yyy', 315], ['zzz', 171]]

It seems like set in python does not work for it.

mylist = list(set(mylist))

Is there any fast and easy way of doing this in python (perhaps using libraries)?

Upvotes: 3

Views: 9043

Answers (4)

John Joseph Fernandes
John Joseph Fernandes

Reputation: 369

The reason that you're not able to do this is because you have a list of lists and not a list of tuples.

What you could do is:

mytuplelist = [tuple(item) for item in mylist]
mylist = list(set(mytuplelist))

or 

mylist = list(set(map(tuple, mylist)))

Upvotes: 6

cs95
cs95

Reputation: 402483

You need to write code that keeps the first of the sub-lists, dropping the rest. The simplest way to do this is to reverse mylist, load it into an dict object, and retrieve its key-value pairs as lists again. 

>>> list(map(list, dict(mylist).items()))

Or, using a list comprehension -

>>> [list(v) for v in dict(mylist).items()]


[['zzz', 171], ['yyy', 315], ['xxx', 879]]

Note, that this answer does not maintain order! Also, if your sub-lists can have more than 2 elements, an approach involving hashing the tuplized versions of your data, as @JohnJosephFernandez' answer shows, would be the best thing to do.

Upvotes: 5

RoadRunner
RoadRunner

Reputation: 26315

It seems like you want to preserve order. In that case you can keep a set that keeps track of what lists have been added.

Here is an example:

mylist = [['xxx', 879], ['yyy', 315], ['xxx', 879], ['zzz', 171], ['yyy', 315]]

# set that keeps track of what elements have been added
seen = set()

no_dups = []
for lst in mylist:

    # convert to hashable type
    current = tuple(lst)

    # If element not in seen, add it to both
    if current not in seen:
        no_dups.append(lst)
        seen.add(current)

print(no_dups)

Which Outputs:

[['xxx', 879], ['yyy', 315], ['zzz', 171]]

Note: Since lists are not hashable, you can add tuples instead to the seen set.

Upvotes: 6

Jonathon McMurray
Jonathon McMurray

Reputation: 2981

Another option:

>>> mylist = [['xxx', 879], ['yyy', 315], ['xxx', 879], ['zzz', 171], ['yyy', 315]]
>>> y = []
>>> for x in mylist:
...     if not x in y:
...             y+=[x]
...
>>> y
[['xxx', 879], ['yyy', 315], ['zzz', 171]]

Upvotes: 2

Related Questions