CODEWITHSUNDEEP
pythonlist
lofidevops
lofidevops

Reputation: 16962

Is there a more Pythonic way to prevent adding a duplicate to a list?

Is there a more Pythonic (or succinct) way to prevent adding a duplicate to a list?

if item not in item_list:
    item_list.append(item)

Or is this in fact a cheap operation?

Upvotes: 10

Views: 42639

Answers (6)

Rik Schoonbeek
Rik Schoonbeek

Reputation: 4460

For when you have objects in a list and need to check a certain attribute to see if it's already in the list.

Not saying this is the best solution, but it does the job:

    def _extend_object_list_prevent_duplicates(list_to_extend, sequence_to_add, unique_attr):
        """
        Extends list_to_extend with sequence_to_add (of objects), preventing duplicate values. Uses unique_attr to distinguish between objects.
        """
        objects_currently_in_list = {getattr(obj, unique_attr) for obj in list_to_extend}
        for obj_to_add in sequence_to_add:
            obj_identifier = getattr(obj_to_add, unique_attr)
            if obj_identifier not in objects_currently_in_list:
                list_to_extend.append(obj_to_add)
        return list_to_extend

Upvotes: 0

brc
brc

Reputation: 269

If you have multiple places where you append to the collection its not very convenient to write boilerplate code like if item not in item_list:.... , you either should have a separate function that tracks changes to collection or subclass list with 'append' method override:

class CollisionsList(list):
    def append(self, other):
        if other in self:
            raise ValueError('--> Value already added: {0}'.format(other))
        super().append(other)


l = CollisionsList()
l.append('a')
l.append('b')
l.append('a')
print(l)

Upvotes: 3

Chris Hagmann
Chris Hagmann

Reputation: 1096

Your way is great! Set are useful for this sort of things but as previously mentioned, they don't maintain order. Other ways of writing a little more succinctly, though maybe not as clear, are show below:

item_list.append(item) if item not in item_list else None

and

item_list += [item] if item not in item_list else []

this last one can be adapted if you wanted to add multiple new_items = [item1, ...] like so

item_list += [item for item in new_items if item not in item_list]

Upvotes: 4

johnsyweb
johnsyweb

Reputation: 141800

Since @hcwsha's original solution has been replaced, I'm recording it here:

seen = set(item_list)

# [...]

if item not in seen:
    seen.add(item)
    item_list.append(item)

This runs in O (1) and could therefore be considered better than the one that you are currently using.

Upvotes: 18

Ashwini Chaudhary
Ashwini Chaudhary

Reputation: 250951

Use a set to keep track of seen items, sets provide O(1) lookup. 

>>> item_list = [1, 7, 7, 7, 11, 14 ,100, 100, 4, 4, 4]
>>> seen = set()
>>> item_list[:] = [item for item in item_list
                                       if item not in seen and not seen.add(item)]
>>> item_list
[1, 7, 11, 14, 100, 4]

If order doesn't matter then simply use set() on item_list:

>>> set(item_list)
set([1, 100, 7, 11, 14, 4])

Upvotes: 3

Deelaka
Deelaka

Reputation: 13693

You can use the built-in set() function as shown below and the list() function to convert that set object to a normal python list:

item_list = ['a','b','b']

print list(set(item_list))
#['a', 'b']

Note: The order is not maintained when using sets

Upvotes: 0

Related Questions