CODEWITHSUNDEEP
pythondictionary
AGH_TORN
AGH_TORN

Reputation: 833

Combining multiple lists of dictionaries

I have several lists of dictionaries, where each dictionary contains a unique id value that is common among all lists. I'd like to combine them into a single list of dicts, where each dict is joined on that id value.

list1 = [{'id': 1, 'value': 20}, {'id': 2, 'value': 21}]
list2 = [{'id': 1, 'sum': 10}, {'id': 2, 'sum': 11}]
list3 = [{'id': 1, 'total': 30}, {'id': 2, 'total': 32}]

desired_output = [{'id': 1, 'value': 20, 'sum': 10, 'total': 30}, {'id': 2, 'value': 21, 'sum': 11, 'total': 32}]

I tried doing something like the answer found at https://stackoverflow.com/a/42018660/7564393, but I'm getting very confused since I have more than 2 lists. Should I try using a defaultdict approach? More importantly, I am NOT always going to know the other values, only that the id value is present in all dicts.

Upvotes: 2

Views: 105

Answers (6)

misha
misha

Reputation: 122

list1 = [{'id': 1, 'value': 20}, {'id': 2, 'value': 21}]
list2 = [{'id': 1, 'sum': 10}, {'id': 2, 'sum': 11}]
list3 = [{'id': 1, 'total': 30}, {'id': 2, 'total': 32}]

result = []
for i in range(0,len(list1)):
      final_dict = dict(list(list1[i].items()) + list(list2[i].items()) + list(list3[i].items()))
      result.append(final_dict)

print(result)

output : [{'id': 1, 'value': 20, 'sum': 10, 'total': 30}, {'id': 2, 'value': 21, 'sum': 11, 'total': 32}]

Upvotes: 0

hygull
hygull

Reputation: 8740

Here, I have presented a functional approach without using itertools (which is excellent in rapid development work).

This solution will work for any number of lists as the function takes variable number of arguments and also let user to specify the type of return output (list/dict).

By default it returns list as you want that otherwise it returns dictionary in case if you pass as_list = False.

I preferred dictionary to solve this because its fast and search complexity is also less.

Just have a look at the below get_packed_list() function.

get_packed_list()

def get_packed_list(*dicts_lists, as_list=True):
    output = {}

    for dicts_list in dicts_lists:
        for dictionary in dicts_list:
            _id = dictionary.pop("id") # id() is in-built function so preferred _id

            if _id not in output:
                # Create new id
                output[_id] = {"id": _id}

            for key in dictionary:
                output[_id][key] = dictionary[key]

            dictionary["id"] = _id # push back the 'id' after work (call by reference mechanism)

    if as_list:
        return [output[key] for key in output]

    return output # dictionary

Test

list1 = [{'id': 1, 'value': 20}, {'id': 2, 'value': 21}]
list2 = [{'id': 1, 'sum': 10}, {'id': 2, 'sum': 11}]
list3 = [{'id': 1, 'total': 30}, {'id': 2, 'total': 32}]


output = get_packed_list(list1, list2, list3)
print(output) 
# [{'id': 1, 'value': 20, 'sum': 10, 'total': 30}, {'id': 2, 'value': 21, 'sum': 11, 'total': 32}]

output = get_packed_list(list1, list2, list3, as_list=False)
print(output)
# {1: {'id': 1, 'value': 20, 'sum': 10, 'total': 30}, 2: {'id': 2, 'value': 21, 'sum': 11, 'total': 32}}

Upvotes: 1

user12101695
user12101695

Reputation: 

list1 = [{'id': 1, 'value': 20}, {'id': 2, 'value': 21}]
list2 = [{'id': 1, 'sum': 10}, {'id': 2, 'sum': 11}]
list3 = [{'id': 1, 'total': 30}, {'id': 2, 'total': 32}]

print(list1+list2+list3)

Upvotes: 0

Thierry Lathuille
Thierry Lathuille

Reputation: 24232

You can first build a dict of dicts, then turn it into a list:

from itertools import chain
from collections import defaultdict

list1 = [{'id': 1, 'value': 20}, {'id': 2, 'value': 21}]
list2 = [{'id': 1, 'sum': 10}, {'id': 2, 'sum': 11}]
list3 = [{'id': 1, 'total': 30}, {'id': 2, 'total': 32}]

dict_out = defaultdict(dict)

for d in chain(list1, list2, list3):
    dict_out[d['id']].update(d)
out = list(dict_out.values())

print(out)
# [{'id': 1, 'value': 20, 'sum': 10, 'total': 30}, {'id': 2, 'value': 21, 'sum': 11, 'total': 32}]

itertools.chain allows you to iterate on all the dicts contained in the 3 lists. We build a dict dict_out having the id as key, and the corresponding dict being built as value. This way, we can easily update the already built part with the small dict of our current iteration.

Upvotes: 1

Olvin Roght
Olvin Roght

Reputation: 7812

You can use itertools.groupby():

from itertools import groupby

list1 = [{'id': 1, 'value': 20}, {'id': 2, 'value': 21}]
list2 = [{'id': 1, 'sum': 10}, {'id': 2, 'sum': 11}]
list3 = [{'id': 1, 'total': 30}, {'id': 2, 'total': 32}]

desired_output = []
for _, values in groupby(sorted([*list1, *list2, *list3], key=lambda x: x['id']), key=lambda x: x['id']):
    temp = {}
    for d in values:
        temp.update(d)
    desired_output.append(temp)

Result:

[{'id': 1, 'value': 20, 'sum': 10, 'total': 30}, {'id': 2, 'value': 21, 'sum': 11, 'total': 32}]

Upvotes: 1

Mohammad Zeineldeen
Mohammad Zeineldeen

Reputation: 312

list1 = [{'id': 1, 'value': 20}, {'id': 2, 'value': 21}]
list2 = [{'id': 1, 'sum': 10}, {'id': 2, 'sum': 11}]
list3 = [{'id': 1, 'total': 30}, {'id': 2, 'total': 32}]

# combine all lists
d = {} # id -> dict
for l in [list1, list2, list3]:
    for list_d in l:
        if 'id' not in list_d: continue
        id = list_d['id']
        if id not in d:
            d[id] = list_d
        else:
            d[id].update(list_d)

# dicts with same id are grouped together since id is used as key
res = [v for v in d.values()]
print(res)

Upvotes: 1

Related Questions