CODEWITHSUNDEEP
pythondictionary
Arjun
Arjun

Reputation: 31

Remove duplicate from the value of particular dictionary key in Python

I need to remove duplicate values from the particular key in dict for example

I'm have 

data = [{'NAME':'John','AGE':23,'NUMBER':345},
        {'NAME':'Michel','AGE':23,'NUMBER':346},
        {'NAME':'RAHUL','AGE':23,'NUMBER':347},
        {'NAME':'Susea','AGE':23,'NUMBER':346},
        {'NAME':'Wincent','AGE':23,'NUMBER':342}]

In above i need to unique the 'NUMBER' key as unique Note: {'NUMBER':346} occurs 2 times. I need output as 

data = [{'NAME':'John','AGE':23,'NUMBER':'345'},
        {'NAME':'Michel','AGE':23,'NUMBER':346},
        {'NAME':'RAHUL','AGE':23,'NUMBER':347},
        {'NAME':'Wincent','AGE':23,'NUMBER':342}]

(i.e) removing any of the duplicate record for particular key duplicate

Pls help me

Upvotes: 0

Views: 71

Answers (4)

canerbalci
canerbalci

Reputation: 1297

Something like this:

data = [{'NAME':'John','AGE':23,'NUMBER':345},
        {'NAME':'Michel','AGE':23,'NUMBER':346},
        {'NAME':'RAHUL','AGE':23,'NUMBER':347},
        {'NAME':'Susea','AGE':23,'NUMBER':346},
        {'NAME':'Wincent','AGE':23,'NUMBER':342}]

filtered_data = []
seen = set()
for item in data:
    number = item['NUMBER']
    if number not in seen:
        filtered_data.append(item)
        seen.add(number)
# filtered_data is deduped

Upvotes: 1

ElectricTech
ElectricTech

Reputation: 17

Here are some steps: 1. Create an empty list for your found_numbers. 2. Iterate over all dictionary entries. 3. For each dictionary, look to see if the value for the number key is in your found_numbers list. If the number exists this is a duplicate entry. You can remove it from your list Else the number is not found so this is the first instance of a dictionary with this number. We need to add this number to the found_numbers list.

Note: This assumes a first come first serve attitude towards duplicate entries.

Upvotes: 0

Mike Müller
Mike Müller

Reputation: 85442

This works:

key = 'NUMBER'
seen = set()
res = []
for entry in data:
    if not entry[key] in seen:
        res.append(entry)
        seen.add(entry[key])

Result:

>>> res
[{'AGE': 23, 'NAME': 'John', 'NUMBER': 345},
 {'AGE': 23, 'NAME': 'Michel', 'NUMBER': 346},
 {'AGE': 23, 'NAME': 'RAHUL', 'NUMBER': 347},
 {'AGE': 23, 'NAME': 'Wincent', 'NUMBER': 342}]

Use a set to hold the already seen values. For large lists this is much more efficient than using a list such as val in seen_list.

Upvotes: 0

Vivek Sable
Vivek Sable

Reputation: 10213

Create unique list of Number.

Algo:

  1. uni_no is unique number list.
  2. data1 final output of filter process.
  3. Iterate on data by for loop
  4. check if NUMBER is present iterator item.
  5. If not then add to data1 and add Number to uni_no

Demo:

>>> data = [{'NAME':'John','AGE':23,'NUMBER':345},
...         {'NAME':'Michel','AGE':23,'NUMBER':346},
...         {'NAME':'RAHUL','AGE':23,'NUMBER':347},
...         {'NAME':'Susea','AGE':23,'NUMBER':346},
...         {'NAME':'Wincent','AGE':23,'NUMBER':342}]
>>> 
>>> data
[{'AGE': 23, 'NAME': 'John', 'NUMBER': 345}, {'AGE': 23, 'NAME': 'Michel', 'NUMBER': 346}, {'AGE': 23, 'NAME': 'RAHUL', 'NUMBER': 347}, {'AGE': 23, 'NAME': 'Susea', 'NUMBER': 346}, {'AGE': 23, 'NAME': 'Wincent', 'NUMBER': 342}]
>>> uni_no = []
>>> data1 = []
>>> for i in data:
...     if i["NUMBER"] not in uni_no:
...         uni_no.append(i["NUMBER"] )
...         data1.append(i)
... 
>>> data1
[{'AGE': 23, 'NAME': 'John', 'NUMBER': 345}, {'AGE': 23, 'NAME': 'Michel', 'NUMBER': 346}, {'AGE': 23, 'NAME': 'RAHUL', 'NUMBER': 347}, {'AGE': 23, 'NAME': 'Wincent', 'NUMBER': 342}]

Upvotes: 0

Related Questions