CODEWITHSUNDEEP
pythondictionarygrouping
goRunToStack
goRunToStack

Reputation: 253

Grouping python list of dictionaries and aggregation value data

I have input list

inlist = [{"id":123,"hour":5,"groups":"1"},{"id":345,"hour":3,"groups":"1;2"},{"id":65,"hour":-2,"groups":"3"}]

I need to group the dictionaries by 'groups' value. After that I need to add key min and max of hour in new grouped lists. The output should look like this

outlist=[(1, [{"id":123, "hour":5, "min_group_hour":3, "max_group_hour":5}, {"id":345, "hour":3, "min_group_hour":3, "max_group_hour":5}]),
     (2, [{"id":345, "hour":3, "min_group_hour":3, "max_group_hour":3}])
     (3, [{"id":65, "hour":-2, "min_group_hour":-2, "max_group_hour":-2}])]

So far I managed to group input list

new_list = []
for domain in test:
    for group in domain['groups'].split(';'):
        d = dict()
        d['id'] = domain['id']
        d['group'] = group
        d['hour'] = domain['hour']
        new_list.append(d)

for k,v in itertools.groupby(new_list, key=itemgetter('group')):
    print (int(k),max(list(v),key=itemgetter('hour'))

And output is

('1', [{'group': '1', 'id': 123, 'hour': 5}])
('2', [{'group': '2', 'id': 345, 'hour': 3}])
('3', [{'group': '3', 'id': 65, 'hour': -2}])

I don't know how to aggregate values by group? And is there more pythonic way of grouping dictionaries by key value that needs to be splitted?

Upvotes: 2

Views: 1489

Answers (2)

Abbas
Abbas

Reputation: 4070

IIUC: Here is another way to do it in pandas:

import pandas as pd

input = [{"id":123,"hour":5,"group":"1"},{"id":345,"hour":3,"group":"1;2"},{"id":65,"hour":-2,"group":"3"}]
df = pd.DataFrame(input)
#Get minimum
dfmi = df.groupby('group').apply(min)
#Rename hour column as min_hour
dfmi.rename(columns={'hour':'min_hour'}, inplace=True)
dfmx = df.groupby('group').apply(max)
#Rename hour column as max_hour
dfmx.rename(columns={'hour':'max_hour'}, inplace=True)
#Merge min df with main df
df = df.merge(dfmi, on='group', how='outer')
#Merge max df with main df
df = df.merge(dfmx, on='group', how='outer')
output = list(df.apply(lambda x: x.to_dict(), axis=1))
#Dictionary of dictionaries
dict_out = df.to_dict(orient='index')

Upvotes: 1

Aran-Fey
Aran-Fey

Reputation: 43126

Start by creating a dict that maps group numbers to dictionaries:

from collections import defaultdict

dicts_by_group = defaultdict(list)
for dic in inlist:
    groups = map(int, dic['groups'].split(';'))
    for group in groups:
        dicts_by_group[group].append(dic)

This gives us a dict that looks like

{1: [{'id': 123, 'hour': 5, 'groups': '1'},
     {'id': 345, 'hour': 3, 'groups': '1;2'}],
 2: [{'id': 345, 'hour': 3, 'groups': '1;2'}],
 3: [{'id': 65, 'hour': -2, 'groups': '3'}]}

Then iterate over the grouped dicts and set the min_group_hour and max_group_hour for each group:

outlist = []
for group in sorted(dicts_by_group.keys()):
    dicts = dicts_by_group[group]
    min_hour = min(dic['hour'] for dic in dicts)
    max_hour = max(dic['hour'] for dic in dicts)

    dicts = [{'id': dic['id'], 'hour': dic['hour'], 'min_group_hour': min_hour,
              'max_group_hour': max_hour} for dic in dicts]
    outlist.append((group, dicts))

Result:

[(1, [{'id': 123, 'hour': 5, 'min_group_hour': 3, 'max_group_hour': 5},
      {'id': 345, 'hour': 3, 'min_group_hour': 3, 'max_group_hour': 5}]),
 (2, [{'id': 345, 'hour': 3, 'min_group_hour': 3, 'max_group_hour': 3}]),
 (3, [{'id': 65, 'hour': -2, 'min_group_hour': -2, 'max_group_hour': -2}])]

Upvotes: 2

Related Questions