CODEWITHSUNDEEP
python
AlexW
AlexW

Reputation: 2587

Merge a single list of dictionaries with the same key value

I can see many similar questions on here but not one that I can find that helps me get the desired output.

I have a single list of dictionaries that have the same ID but with different key value pairs, id like to join all those key values pairs into a single list entry, below is a sample of data and the desired output.

Thanks for your help

data = [
    {'id': '10', 'animal' : 'cat'},
    {'id': '11', 'animal' : 'dog'},
    {'id': '3', 'animal' : 'pigeon'},
    {'id': '10', 'color' : 'yellow'},
    {'id': '11', 'color' : 'brown'},
    {'id': '3', 'color' : 'grey'},
    {'id': '10', 'type' : 'furry'},
    {'id': '11', 'type' : 'fluffy'},
    {'id': '3', 'type' : 'dirty'},
]

desired output

data = [
    {'id': '10', 'animal' : 'cat', 'color' : 'yellow', 'type' : 'furry'},
    {'id': '11', 'animal' : 'dog', 'color' : 'brown', 'type' : 'fluffy'},
    {'id': '3', 'animal' : 'pigeon', 'color' : 'grey', 'type' : 'dirty'},
]

Upvotes: 1

Views: 1114

Answers (3)

chepner
chepner

Reputation: 532418

In Python 3.9 (ETA Fall 2020), you'll be able to use the | operator to merge dicts with the same id key.

from itertools import groupby
from operator import or_, itemgetter
from functools import reduce

# I know *why* groupby doesn't have an option to
# sort your data first, but that doesn't mean I can't
# wish that it could...
def group(data, key):
    "Iterate over groups of dicts considered equal according to key"
    yield from map(itemgetter(1), groupby(sorted(data, key=key), key))

data = [
    {'id': '10', 'animal' : 'cat'},
    {'id': '11', 'animal' : 'dog'},
    {'id': '3', 'animal' : 'pigeon'},
    {'id': '10', 'color' : 'yellow'},
    {'id': '11', 'color' : 'brown'},
    {'id': '3', 'color' : 'grey'},
    {'id': '10', 'type' : 'furry'},
    {'id': '11', 'type' : 'fluffy'},
    {'id': '3', 'type' : 'dirty'},
    ]


# E.g., {'id': 10, 'animal': 'cat'} | {'id': 10, 'color': 'yellow'}
#  == {'id': 10, 'animal': 'cat', 'color': 'yellow'}
data = [reduce(or_, ds) for ds in group(data, itemgetter('id'))]

Upvotes: 0

Praveen
Praveen

Reputation: 9365

You can do it using groupby and ChainMap

from itertools import groupby
from collections import ChainMap

id_getter = lambda x: x['id']
gp = groupby(sorted(data, key=id_getter), key=id_getter)
result = [dict(ChainMap(*a)) for _, a in gp]

groupby works on sorted collection, hence sort data before call groupby
ChainMap is used to merge a list of dictionaries to a single dictionary

Upvotes: 1

AdamGold
AdamGold

Reputation: 5071

There are multiple ways to achieve this, one of which is defaultdict:

In [1]: data = [
   ...:     {'id': '10', 'animal' : 'cat'},
   ...:     {'id': '11', 'animal' : 'dog'},
   ...:     {'id': '3', 'animal' : 'pigeon'},
   ...:     {'id': '10', 'color' : 'yellow'},
   ...:     {'id': '11', 'color' : 'brown'},
   ...:     {'id': '3', 'color' : 'grey'},
   ...:     {'id': '10', 'type' : 'furry'},
   ...:     {'id': '11', 'type' : 'fluffy'},
   ...:     {'id': '3', 'type' : 'dirty'},
   ...: ]

In [2]: from collections import defaultdict
   ...: ids = defaultdict(dict)
   ...: for d in data:
   ...:     ids[d["id"]].update(d)
   ...:


In [6]: list(ids.values())
Out[6]:
[{'id': '10', 'animal': 'cat', 'color': 'yellow', 'type': 'furry'},
 {'id': '11', 'animal': 'dog', 'color': 'brown', 'type': 'fluffy'},
 {'id': '3', 'animal': 'pigeon', 'color': 'grey', 'type': 'dirty'}]

Upvotes: 2

Related Questions