CODEWITHSUNDEEP
pythonlistduplicates
stefanbschneider
stefanbschneider

Reputation: 6086

Removing duplicates from list: Keeping the order and the last duplicate of each element

I have a ordered list with duplicates (objects that compare equal) and I want to remove the duplicates. As I want to keep the order, I cannot use set.

The duplicates in my list don't ocurr directly after each other and I want to keep the last ocurrence. In this related post, it seems like only the first ocurrence of a duplicate is kept.

As I simplified example, this is what I want:

list_with_duplicates = [1, 2, 1, 3, 2, 1]
list_without_duplicates = [3, 2, 1]

The only thing I can think of is a messy implementation using multiple iterations to first save the last ocurrence of each duplicate and then recreating the original list using the order of the last ocurring duplicates.

Upvotes: 0

Views: 3897

Answers (5)

ekhumoro
ekhumoro

Reputation: 120608

For Python >= 3.7, dict is ordered, so a simple solution is:

>>> x = [1, 2, 1, 3, 2, 1]
>>> list(reversed(dict.fromkeys(reversed(x))))
[3, 2, 1]

or:

>>> list(dict.fromkeys(x[::-1]))[::-1]
[3, 2, 1]

For earlier Python versions, use OrderedDict instead of dict.

Upvotes: 2

Mayank Porwal
Mayank Porwal

Reputation: 34056

This could be a clean solution too:

x = [1, 2, 1, 3, 2, 1]
sorted(set(x), key=lambda i: x[::-1].index(i), reverse=True)
# [3, 2, 1]

Upvotes: 1

yihuang
yihuang

Reputation: 329

>>> def keep_second(x, y):
>>>     return y
>>> [functools.reduce(keep_second, group) for _, group in itertools.groupby(sorted(items, key=key_func), key=key_func)

Upvotes: 0

TigerhawkT3
TigerhawkT3

Reputation: 49318

Use any answer you like in the linked duplicate (such as the top one), with one change: reverse your list before you iterate over it, and reverse the result when you're done.

def rem_rev(seq):
    seen = set()
    seen_add = seen.add
    return [x for x in seq[::-1] if not (x in seen or seen_add(x))][::-1]
    #                     ^^^^^^                                   ^^^^^^
    #                  or reversed(seq)

Upvotes: 2

Maurice Meyer
Maurice Meyer

Reputation: 18106

You can reverse a list using '::-1':

>>> result = []
>>> for item in l[::-1]:
...     if item not in result:
...             result.append(item)
... 
>>> result[::-1]
[3, 2, 1]
>>>

Upvotes: 0

Related Questions