CODEWITHSUNDEEP
pythonarrays
Chris Hall
Chris Hall

Reputation: 931

Remove Duplicates in two-dimensional array while maintaining sequence

I have found a lot of threads on removing duplicates in arrays but none for my specific use-case. I have a two-dimensional list that I need to remove duplicates from however I must maintain the original sequence

mylist = [['Installation', '64%'], ['C2', '14%'], ['NA', '14%'], ['C2', '14%'], ['NA', '14%'], ['na', '7%']]

I need to simply drop the duplicates without re-arranging, so..

newlist = [['Installation', '64%'], ['C2', '14%'], ['NA', '14%'], ['na', '7%']]

appreciate any help

Upvotes: 3

Views: 13966

Answers (3)

Afridi
Afridi

Reputation: 41

import numpy as np :

myList = [['Installation', '64%'], ['C2', '14%'], ['NA', '14%'], ['C2', '14%'], ['NA', '14%'], ['na', '7%']]

mylist=np.unique(myList,axis=0)

print (mylist)

Upvotes: 4

Andrea de Marco
Andrea de Marco

Reputation: 825

mylist = [['Installation', '64%'], ['C2', '14%'], ['NA', '14%'], ['C2', '14%'], ['NA', '14%'], ['na', '7%']]
result = []
for x in mylist:
    if x not in result:
        result.append(x)
print result

[['Installation', '64%'], ['C2', '14%'], ['NA', '14%'], ['na', '7%']]

Upvotes: 5

falsetru
falsetru

Reputation: 369334

Using set to keep track of seen items:

>>> mylist = [['Installation', '64%'], ['C2', '14%'], ['NA', '14%'], ['C2', '14%'], ['NA', '14%'], ['na', '7%']]
>>> seen = set()
>>> newlist = []
>>> for item in mylist:
...     t = tuple(item)
...     if t not in seen:
...         newlist.append(item)
...         seen.add(t)
...
>>> newlist
[['Installation', '64%'], ['C2', '14%'], ['NA', '14%'], ['na', '7%']]

NOTE

You need to convert a list to tuple (list is not hashable); can't add a list to set.

>>> seen = set()
>>> seen.add([1,2])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'
>>> seen.add(tuple([1,2]))
>>>

Upvotes: 8

Related Questions