How can I update a list of dictionaries here?

Question

I am trying to update listy with values from updated; each of which are lists of dictionaries. For each value in updated, if the 'name' key and 'coverage' key match a dictionary in listy, then I need to update it. If not, then simply append that value to listy. The order of 'coverage' doesn't matter; so long as the lists match.

listy = [{'name':'Dan','coverage':[{'city':'Malibu','state':'California'}],'employees':'12'},
           {'name':'Emily','coverage':[{'city':'Boston','state':'Massachusetts'}],'employees':'8'},
           {'name':'Martin','coverage':[{'city':'Malibu','state':'California'},{'city':'San Antonio','state':'Texas'}],'employees':'5'},
           {'name':'Steve','coverage':[{'city':'Madison','state':'Wisconsin'},{'city':'Denver','state':'Colorado'}],'employees':'4'},
           {'name':'Steve','coverage':[{'city':'Madison','state':'Wisconsin'}],'employees':'3'},
           {'name':'Iris','coverage':[{'city':'Boise','state':'Idaho'}],'employees':'0'},
           {'name':'Axl','coverage':[{'city':'Omaha','state':'Nebraska'}],'employees':'4'},
           {'name':'Iris','coverage':[{'city':'New York','state':'New York'}],'employees':'7'},
           {'name':'Floyd','coverage':[{'city':'Denver','state':'Colorado'}],'employees':'3'},
           {'name':'Martin','employees':'5'}
           ]

  updated = [
             {'name':'Emily','coverage':[{'city':'Boston','state':'Massachusetts'}],'employees':'5'},
             {'name':'Dan','coverage':[{'city':'Malibu','state':'California'}],'employees':'13'},
             {'name':'Martin','coverage':[{'city':'San Antonio','state':'Texas'},{'city':'Malibu','state':'California'}],'employees':'7'},
             {'name':'Steve','coverage':[{'city':'Madison','state':'Wisconsin'}],'employees':'6'},
             {'name':'Steve','coverage':[{'city':'Chicago','state':'Illinois'}],'employees':'3'},
             {'name':'Steve','coverage':[{'city':'Denver','state':'Colorado'},{'city':'Madison','state':'Wisconsin'}],'employees':'8'},
             {'name':'Randall','coverage':[{'city':'Balmont','state':'Virginia'}],'employees':'12'},
             {'name':'Rachel','employees':'1'}
          ]




for u in updated:
    if 'coverage' in u:
      u['coverage'].sort()
      print 'Looking for:', u['name'],u['coverage']
      match = [x for x in listy if 'coverage' in x if x['name']==u['name'] and x['coverage'].sort()==u['coverage'].sort()]
    else:
      print 'Looking for:', u['name']
      match = [x for x in listy if 'coverage' not in x and x['name'] == u['name']]
    print 'Found:', match
    print '-------'

The above prints:

Looking for: Emily [{'city': 'Boston', 'state': 'Massachusetts'}]
Found: [{'employees': '8', 'name': 'Emily', 'coverage': [{'city': 'Boston', 'state': 'Massachusetts'}]}]
-------
Looking for: Dan [{'city': 'Malibu', 'state': 'California'}]
Found: [{'employees': '12', 'name': 'Dan', 'coverage': [{'city': 'Malibu', 'state': 'California'}]}]
-------
Looking for: Martin [{'city': 'Malibu', 'state': 'California'}, {'city': 'San Antonio', 'state': 'Texas'}]
Found: [{'employees': '5', 'name': 'Martin', 'coverage': [{'city': 'Malibu', 'state': 'California'}, {'city': 'San Antonio', 'state': 'Texas'}]}]
-------
Looking for: Steve [{'city': 'Madison', 'state': 'Wisconsin'}]
Found: [{'employees': '4', 'name': 'Steve', 'coverage': [{'city': 'Denver', 'state': 'Colorado'}, {'city': 'Madison', 'state': 'Wisconsin'}]}, {'employees': '3', 'name': 'Steve', 'coverage': [{'city': 'Madison', 'state': 'Wisconsin'}]}]
-------
Looking for: Steve [{'city': 'Chicago', 'state': 'Illinois'}]
Found: [{'employees': '4', 'name': 'Steve', 'coverage': [{'city': 'Denver', 'state': 'Colorado'}, {'city': 'Madison', 'state': 'Wisconsin'}]}, {'employees': '3', 'name': 'Steve', 'coverage': [{'city': 'Madison', 'state': 'Wisconsin'}]}]
-------
Looking for: Steve [{'city': 'Denver', 'state': 'Colorado'}, {'city': 'Madison', 'state': 'Wisconsin'}]
Found: [{'employees': '4', 'name': 'Steve', 'coverage': [{'city': 'Denver', 'state': 'Colorado'}, {'city': 'Madison', 'state': 'Wisconsin'}]}, {'employees': '3', 'name': 'Steve', 'coverage': [{'city': 'Madison', 'state': 'Wisconsin'}]}]
-------
Looking for: Randall [{'city': 'Balmont', 'state': 'Virginia'}]
Found: []
-------
Looking for: Rachel
Found: []
-------

As you can see, Steve is a bit messed up. It will pick up every 'Steve' in listy even if the 'coverage' doesn't match (even though I've specified that they should match). Even if I specify len(x['coverage'])==len(u['coverage']) in the list comprehension it still is messed up. What am I doing wrong? Also, even if I do correct the error(s) I have I have no idea how I will actually update listy with the new value.

Answer 1

One problem is that sort() operates on a list in-place and returns None, so this does not do what you probably intend:

x['coverage'].sort() == u['coverage'].sort()

See sorted() instead.

A few other ideas and suggestions:

CV = 'coverage'
NM = 'name'

# If possible, make sure all of the data meets basic expectations.
# This simplifies other logic.
def normalize(bosses):
    for b in bosses:
        if CV not in b: b[CV] = []
        if NM not in b: b[NM] = None
        b[CV].sort()

normalize(listy)
normalize(updated)

def bosses_match(b1, b2):
    return b1[NM] == b2[NM] and b1[CV] == b2[CV]

for u in updated:
    matches = [i for i, x in enumerate(listy) if bosses_match(u, x)]
    if matches:
        for i in matches:
            listy[i] = u
    else:
        listy.append(u)

Note that your approach will be very slow if your lists are large, because you are searching your main list over and over. A better approach would be to use a data structure that would allow you to quickly lookup matching items. At a minimum, you could organized the data by the name. Then your search for matches would operate over a much smaller space. For example:

from collections import defaultdict

lookup = defaultdict(list)
for i, b in enumerate(listy):
    lookup[b[NM]].append(i)     # Store the item, or the index.