CODEWITHSUNDEEP
pythonpython-2.7dictionary
GreetRufus
GreetRufus

Reputation: 431

Compare Two Lists of Dictionaries and Output diffs

I have 2 lists of dicts that I need to compare and output the difference for a report

I need to:

  1. Check new_list against old_list for new records (records that exist in new_list but not in old_list) and append these to updated_list
  2. Ignore matching records (all key, values match)
  3. Find matching records with different event15 or event16 and update the event with the diff of the 2 values
  4. Output results to a new list of dicts (updated_list in this example)

To process:

new_list = [
{'datetime': '2018-08-01', 'evar1': 'newRecord', 'event16': '100', 'event15': '200'},
{'datetime': '2018-08-02', 'evar1': 'duplicateRecord', 'event16': '10', 'event15': '20'},
{'datetime': '2018-08-03', 'evar1': 'diffEvent', 'event16': '15', 'event15': '25'}
]

old_list = [
{'datetime': '2018-08-02', 'evar1': 'duplicateRecord', 'event16': '10', 'event15': '20'},
{'datetime': '2018-08-03', 'evar1': 'diffEvent', 'event16': '10', 'event15': '25'}
]

The results should look like this list:

updated_list = [
{'datetime': '2018-08-01', 'evar1': 'newRecord', 'evar3': 'site',  'event16': '100', 'event15': '200'},
{'datetime': '2018-08-03', 'evar1': 'diffEvent', 'evar3': 'site',  'event16': '5', 'event15': '25'}
]

I tried this:

updated_list = []
for new_item in new_list:
    for old_item in old_list:
        for key, value in new_item.iteritems():
        # If values don't match, subtract old_list value from new_list values and append the diff
        if any(ko == key for ko, vo in old_item.iteritems()):
            ko, vo = [(ko, vo) for (ko, vo) in old_item.iteritems() if ko == key][0]
            if vo != value:
                new_value = value - vo
                new_item.update({ko: new_value})
                updated_list.append(new_item)
            else:
            # If record does not exist in old_list, append the new record
            updated_list.append(new_item)

Upvotes: 1

Views: 80

Answers (1)

Joe Iddon
Joe Iddon

Reputation: 20414

Very confusing task indeed! Here is my solution (comments should explain the method).

#init a list to store the dictionaries
updated_list = []
#define our 'special keys'
events = ('event15', 'event16')
#remove duplicates from both lists (where all key, values match) - case (2)
old_list_no_dupes = [d for d in old_list if not any(d == dd for dd in new_list)]
new_list_no_dupes = [d for d in new_list if not any(d == dd for dd in old_list)]
for d in new_list_no_dupes:
    #iterate over all the dictionaries in old_list for case (3)
    for dd in old_list_no_dupes:
        #continue to next if not every pair (but event*) matches
        if any(k not in dd or dd[k] != v for k,v in d.items() if k not in events):
            continue
        #iterate the to event keys
        for k in events:
            #check both dictionaries have that key and they are different values
            if k in d and k in dd and d[k] != dd[k]:
                #update the new dictionary to be the absolute difference
                d[k] = str(abs(int(dd[k]) - int(d[k])))
    #append our new dictionary - cases (1), (3) and (4)
    updated_list.append(d)

[{'datetime':'2018-08-01', 'evar1':'newRecord', 'event16':'100', 'event15': '200'},
 {'datetime':'2018-08-03', 'evar1':'diffEvent', 'event16':'5',   'event15': '25'}]

Upvotes: 1

Related Questions