CODEWITHSUNDEEP
pythonpython-3.x
Miztory
Miztory

Reputation: 198

Remove duplicate values in list of list python

Trying to remove duplicates in list of list and storing it.

Original List:

list1 = [[a, b, a, b], [b, c, d, c], [a, c, c]]

Looking for output:

list1 = [[a, b], [b,c,d], [a,c]]

My code:

unique_list = []
for sublist in list1:
    element = [elem for elem in sublist if elem not in unique_list]
    if len(element):
        unique_testset.append(element)

My code appends a sublist multiple times and doesn't get rid of the duplicates.

Upvotes: 0

Views: 2469

Answers (4)

Stef
Stef

Reputation: 15504

Removing duplicates the simple way

A classic, efficient way to remove duplicates from a list in python is to build a set from the list: removing duplicates in lists

list_with_dups = [1, 1, 2, 3, 2]
list_without_dups = list(set(list_with_dups))

You can apply this method repeatedly using a list comprehension:

list1 = [['a', 'b', 'a', 'b'], ['b', 'c', 'd', 'c'], ['a', 'c', 'c']]
without_duplicates = [list(set(sublist)) for sublist in list1]
#                  = [['b', 'a'], ['d', 'b', 'c'], ['c', 'a']]

Removing duplicates whilst conserving order

Applying How do you remove duplicates whilst conserving order? to a list of lists:

def f7(seq):
    seen = set()
    seen_add = seen.add
    return [x for x in seq if not (x in seen or seen_add(x))]

list1 = [['a', 'b', 'a', 'b'], ['b', 'c', 'd', 'c'], ['a', 'c', 'c']]
without_duplicates = [f7(sublist) for sublist in list1]
#                  = [['a', 'b'], ['b', 'c', 'd'], ['a', 'c']]

Upvotes: 3

Vinesh
Vinesh

Reputation: 1

Use This code it'll help you

    import numpy as np
    lis=[['a','b','a'], ['b','c','d'], ['c','a','c']]
    uni=np.unique(lis)
    print(uni)

Upvotes: 0

bhavin tandel
bhavin tandel

Reputation: 75

One way you can achieve this is using:

list1 = [[a, b, a, b], [b, c, d, c], [a, c, c]]

for sublist in list1:
   l = list(set(sublist))
   if l:
     unique_list.append(l)

set removes all the duplicate values and we have unique values

Upvotes: 0

Muhammad Rizwan Munawar
Muhammad Rizwan Munawar

Reputation: 692

try this one...

  list1 = [[a, b, a, b], [b, c, d, c], [a, c, c]]
    
  import itertools
    list1.sort()
    list(list1 for list1,_ in itertools.groupby(list1))

Upvotes: 0

Related Questions