Updating values in list of dictionaries

Question

I have a list of dictionaries something like this:

users=[{"name": "David", "team": "reds", "score1": 100, "score2": 20,},
       {"name": "David", "team": "reds", "score1": 20, "score2": 60,},
       {"name": "David", "team": "blues", "score1": 10, "score2": 70,}]

and would really like to get a new processed list of dictionaries something like

summary=[{"team": "reds", "total1": 120, "total2": 80,},
         {"team": "blues", "total1": 120, "total2": 80,}]

preferably looping through the original data just once. I can create a dictionary holding a total value for each user key with this

summary = dict()
for user in users:
   if not user['team'] in summary:
      summary[user['team']]=float(user['score1'])
   else:
      summary[user['team']]+=float(user['score1'])

to give

summary = {'reds': 120,'blues': 10}

but am struggling with producing the list of dictionaries, the nearest I can get is to create a dictionary at the first instance of a team, and then try to append to its values on subsequent occurrences...

summary = []
for user in users:
   if any(d['team'] == user['team'] for d in summary):
      # append to values in the relevant dictionary
      # ??
   else:
      # Add dictionary to list with some initial values
      d ={'team':user['team'],'total1':user['score1'],'total2':user['score2']}
      summary.append(dict(d))

...and it has gotten messy... Am I going about this in completely the wrong way? Can you change values in a dictionary within a list?

Thanks

Answer 1

I think this is good case to use pandas library for python:

>>> import pandas as pd
>>> dfUsers = pd.DataFrame(users)
>>> dfUsers

    name  score1  score2   team
0  David     100      20   reds
1  David      20      60   reds
2  David      10      70  blues

>>> dfUsers.groupby('team').sum()

       score1  score2
team                 
blues      10      70
reds      120      80

And if you really want to put it into dict:

>>> dfRes = dfUsers.groupby('team').sum()
>>> dfRes.columns = ['total1', 'total2']  # if you want to rename columns
>>> dfRes.reset_index().to_dict(orient='records')

[{'team': 'blues', 'total1': 10, 'total2': 70},
 {'team': 'reds', 'total1': 120, 'total2': 80}]

another way to do this is with itertools.groupby:

>>> from itertools import groupby
>>> from operator import itemgetter
>>> users.sort(key=itemgetter('team'))
>>>
>>> res = [{'team': t[0], 'res': list(t[1])} for t in groupby(users, key=itemgetter('team'))]
>>> res = [{'team':t[0], 'total1': sum(x['score1'] for x in t[1]), 'total2': sum(x['score2'] for x in t[1])} for t in res]
>>> res

[{'team': 'blues', 'total1': 10, 'total2': 70},
 {'team': 'reds', 'total1': 120, 'total2': 80}]

Or, if you really want simple python:

>>> res = dict()
>>> for x in users:
       if x['team'] not in res:
           res[x['team']] = [x['score1'], x['score2']]
       else:
           res[x['team']][0] += x['score1']
           res[x['team']][1] += x['score2']
>>> res = [{'team': k, 'total1': v[0], 'total2': v[1]} for k, v in res.iteritems()}]
>>> res

[{'team': 'reds', 'total1': 120, 'total2': 80},
 {'team': 'blues', 'total1': 10, 'total2': 70}]

Answer 2

See comments inline for an explanation

import pprint

users=[{"name": "David", "team": "reds", "score1": 100, "score2": 20,},
       {"name": "David", "team": "reds", "score1": 20, "score2": 60,},
       {"name": "David", "team": "blues", "score1": 10, "score2": 70,}]

scores_by_team = dict()
for user in users:
    if user['team'] not in scores_by_team:
        # Make sure you're gonna have your scores zeroed so you can add the
        # user's scores later
        scores_by_team[user['team']] = {
            'total1': 0,
            'total2': 0
        }
    # Here the user's team exists for sure in scores_by_team
    scores_by_team[user['team']]['total1'] += user['score1']
    scores_by_team[user['team']]['total2'] += user['score2']

# So now, the scores you want have been calculated in a dictionary where the
# keys are the team names and the values are another dictionary with the scores
# that you actually wanted to calculate
print "Before making it a summary: %s" % pprint.pformat(scores_by_team)
summary = list()
for team_name, scores_by_team in scores_by_team.items():
    summary.append(
        {
            'team': team_name,
            'total1': scores_by_team['total1'],
            'total2': scores_by_team['total2'],
        }
    )

print "Summary: %s" % summary

This outputs:

Before making it a summary: {'blues': {'total1': 10, 'total2': 70}, 'reds': {'total1': 120, 'total2': 80}}
Summary: [{'total1': 120, 'total2': 80, 'team': 'reds'}, {'total1': 10, 'total2': 70, 'team': 'blues'}]

Answer 3

Here's my solution which assumes that all scores that need to be added start with score:

users=[{"name": "David", "team": "reds", "score1": 100, "score2": 20,},
       {"name": "David", "team": "reds", "score1": 20, "score2": 60,},
       {"name": "David", "team": "blues", "score1": 10, "score2": 70,}]

totals = {}
for item in users:
    team = item['team']
    if team not in totals:
        totals[team] = {}
    for k,v in item.items():
        if k.startswith('score'):
            if k in totals[team]:
                totals[team][k] += v
            else:
                totals[team][k] = v
print totals

Output:

{'reds': {'score1': 120, 'score2': 80}, 'blues': {'score1': 10, 'score2': 70}}

Answer 4

You are really close, you just need a way to look up which dictionary to update. This is the simplest way I can see.

summary = dict()
for user in users:
   team = user['team']
   if team not in summary:
      summary[team] = dict(team=team,
                           score1=float(user['score1']), 
                           score2=float(user['score2']))
   else:
      summary[team]['score1'] += float(user['score1'])
      summary[team]['score2'] += float(user['score2'])

then

>>> print summary.values()
[{'score1': 120.0, 'score2': 80.0, 'team': 'reds'},
 {'score1': 10.0, 'score2': 70.0, 'team': 'blues'}]