CODEWITHSUNDEEP
pythondictionary
user5594493
user5594493

Reputation: 1130

Merge dicts from a list of dicts based on some key/value pair

I have a list of dicts shown below , I want to merge some dicts into one based some key/value pair.

 [
    {'key': 16, 'value': 3, 'user': 3, 'id': 7}, 
    {'key': 17, 'value': 4, 'user': 3, 'id': 7}, 
    {'key': 17, 'value': 5, 'user': 578, 'id': 7}, 
    {'key': 52, 'value': 1, 'user': 3, 'id': 48}, 
    {'key': 46, 'value': 2, 'user': 578, 'id': 48}
]

Now as you can see dict 1 & 2 have same values for user & id keys. So it is possible to merge these two dicts like 

 [
    {'key': [16,17], 'value': [3,4], 'user': 3, 'id': 7}, 
    {'key': [17], 'value': [5], 'user': 578, 'id': 7}, 
    {'key': [52], 'value': [1], 'user': 3, 'id': 48}, 
    {'key': [46], 'value': [2], 'user': 578, 'id': 48}
]

means user & id value must be unique together.What will be the efficient way to merge (if possible)

Upvotes: 3

Views: 476

Answers (3)

timgeb
timgeb

Reputation: 78690

Let dicts be your original list of dictionaries. This idea maps unique combinations of user and id to defaultdict(list) objects. The final result will be the list of values from that dictionary.

from collections import defaultdict
tmp = defaultdict(dict)

for info in dicts:
    tmp[(info['user'], info['id'])].setdefault('key', []).append(info['key'])
    tmp[(info['user'], info['id'])].setdefault('value', []).append(info['value'])

for (user, id_), d in tmp.items(): # python2: use iteritems
    d.update(dict(user=user, id=id_))

result = list(tmp.values()) # python2: tmp.values() already gives a list
del tmp

Upvotes: 2

Kasravnd
Kasravnd

Reputation: 107287

You can use following aggregate function:

def aggregate(lst):
    new = {}
    for d in lst:
        new.setdefault((d['user'], d['id']), []).append(d)
    for k, d in new.items():
        if len(d) > 1:
            keys, values = zip(*[(sub['key'], sub['value']) for sub in d])
            user, id_ = k
            yield {'key': keys, 'value': values, 'user': user, 'id': id_}
        else:
            yield d[0]

print list(aggregate(lst))
[{'id': 7, 'value': 5, 'key': 17, 'user': 578},
 {'id': 7, 'value': (3, 4), 'key': (16, 17), 'user': 3},
 {'id': 48, 'value': 1, 'key': 52, 'user': 3},
 {'id': 48, 'value': 2, 'key': 46, 'user': 578}]

Upvotes: 0

niemmi
niemmi

Reputation: 17263

Following function will convert the list of dictionaries to new format:

def convert(d):
    res = {}
    for x in d:
        key = (x['user'], x['id'])
        if key in res:
            res[key]['key'].append(x['key'])
            res[key]['value'].append(x['value'])
        else:
            x['key'] = [x['key']]
            x['value'] = [x['value']]
            res[key] = x

    return res.values()

It will mutate the original dictionaries and the ordering of dictionaries in the result will be random. When applied to the input it will produce following result:

[
    {'id': 7, 'value': [5], 'key': [17], 'user': 578}, 
    {'id': 7, 'value': [3, 4], 'key': [16, 17], 'user': 3}, 
    {'id': 48, 'value': [1], 'key': [52], 'user': 3}, 
    {'id': 48, 'value': [2], 'key': [46], 'user': 578}
]

Upvotes: 2

Related Questions