How to avoid nested for loops when combining values from two lists of dictionaries in Python?

Question

Lets say I have two lists:

list1 = [{"sport": 'Soccer',
          "leagues": [{"id": 1001,
                       "name": "League1",
                       "events": [{"id": 100,
                                   "home": "team1",
                                   "away": "team2"},
                                  {"id": 101,
                                   "home": "team3",
                                   "away": "team4"}]}]},
         {"sport": 'Basketball',
          "leagues": [{"id": 1002,
                       "name": "League2",
                       "events": [{"id": 200,
                                  "home": "team5",
                                  "away": "team6"},
                                 {"id": 201,
                                  "home": "team7",
                                  "away": "team8"}]},
                      {"id": 1003,
                       "name": "League3",
                       "events": [{"id": 300,
                                   "home": "team9",
                                   "away": "team10"},
                                  {"id": 301,
                                   "home": "team11",
                                   "away": "team12"}]}],
      }
     ]

list2 = [{"sport": 'Soccer',
          "leagues": [{"id": 1001,
                       "events": [{"id": 100,
                                   "odds": {"home": 1.862, "away": 1.847}},
                                  {"id": 101,
                                   "odds": {"home": 1.70, "away": 2.10}}]}]},
         {"sport": 'Basketball',
          "leagues": [{"id": 1002,
                       "events": [{"id": 200,
                                   "odds": {"home": 1.952, "away": 1.952}},
                                  {"id": 201,
                                   "odds": {"home": 1.90, "away": 2.05}}]},
                      {"id": 1003,
                       "events": [{"id": 300,
                                   "odds": {"home": 1.5, "away": 2.7}},
                                  {"id": 301,
                                   "odds": {"home": 1.75, "away": 2.09}}]}]}]

I would like to combine events from list1 with their respective "odds" from list2. In reality there are many more elements in these two lists, the example is simplified for the sake of clarity. My current (ugly) solution:

for sport_list1 in list1:
    for sport_list2 in list2:
        if sport_list1['sport'] == sport_list2['sport']:
            for league_list1 in sport_list1['leagues']:
                for league_list2 in sport_list2['leagues']:
                    if league_list1['id'] == league_list2['id']:
                        for event_list1 in league_list1['events']:
                            for event_list2 in league_list2['events']:
                                if event_list1['id'] == event_list2['id']:
                                    print(sport_list1['sport'], league_list1['name'], event_list1['home'], event_list1['away'], event_list2['odds'])
                                    break

Desired output:

Soccer League1 team1 team2 {'home': 1.862, 'away': 1.847}
Soccer League1 team3 team4 {'home': 1.7, 'away': 2.1}
Basketball League2 team5 team6 {'home': 1.952, 'away': 1.952}
Basketball League2 team7 team8 {'home': 1.9, 'away': 2.05}
Basketball League3 team9 team10 {'home': 1.5, 'away': 2.7}
Basketball League3 team11 team12 {'home': 1.75, 'away': 2.09}

Any way to make this cleaner and/or more efficient?

Answer 1

@Chronial answer is perfect but here is another approach you might find interesting:

def convert_to_dicts(x):
    if type(x) == list:
        id_field = {"sport", "id"}.intersection(set(x[0].keys())).pop()
        return {y.pop(id_field): convert_to_dicts(y) for y in x}
    elif type(x) == dict:
        return{z: convert_to_dicts(y) for z, y in x.items()}
    return x

def recursive_dict_merge(x, y):
    new_dict = {}
    for key in set(x.keys()).union(set(y.keys())):
        x_val = x.get(key, None)
        y_val = y.get(key, None)
        if type(x_val) == dict and type(y_val) == dict:
            new_dict[key] = recursive_dict_merge(x_val, y_val)
        else:
            new_dict[key] = x_val or y_val
    return new_dict


result = recursive_dict_merge(convert_to_dicts(list1), convert_to_dicts(list2))

I first convert the "list of dict of lists of dicts" into just nested dictionaries.

Then I use recursion to merge those dictionaries.

I think this approach is better because you have an "easy-to-use" result dictionary which then makes it easier to do other things, e.g. the exact print that you want:

for sport, leagues in result.items():
    for league in leagues["leagues"].values():
        for event in league["events"].values():
            print(sport, league['name'], event['home'], event['away'], event['odds'])

In general for problems like this, I find the best first step is usually to reshape the inputs into something more manageable. Nested dictionaries are much easier to think about than lists of dicts of lists of dicts of... etc.

Answer 2

You can write a small helper function. Using dicts frees of you of the O(n²).

def zip_by_key(key, list1, list2):
    map1 = {x[key]: x for x in list1}
    map2 = {x[key]: x for x in list2}
    for k in map1.keys() & map2.keys():
        yield (map1[k], map2[k])

Then your code will be:

for sport1, sport2 in zip_by_key('sport', list1, list2):
    for league1, league2 in zip_by_key('id', sport1['leagues'], sport2['leagues']):
        for event1, event2 in zip_by_key('id', league1['events'], league2['events']):
            print(sport1['sport'], league1['name'], event1['home'], event1['away'], event2['odds'])