CODEWITHSUNDEEP
pythondictionary
Rayne
Rayne

Reputation: 14997

Add new element to inner dictionary in nested dictionary

I'm using Python 3.6.4. I'm trying to create a nested dictionary shared by multiple processes, and add key-value pairs to the inner dictionary depending on the results of various functions.

Below is an example of my code.

manager = Manager()
scores = manager.dict()

def add_scores(name, game):
    global scores
    if name not in scores:
        scores[name] = {}
    if game in scores[name]:
        scores[name][game] += 1
    else:
        scores[name] = {game: 1}

An example of what the populated dictionary would look like is

scores = { 'John': {'golf': '1', 'football': '2'}, 
           'Alice': {'basketball': '1', 'football': '3', 'tennis': 2}}

There are functions for the various games, and depending on the outcome of those functions, a key-value pair would be created for each person.

I know the line scores[name] = {game: 1} is problematic because it just overwrites the scores[name] dictionary with the last game.

I've also tried the code below.

manager = Manager()
scores = manager.dict()

def add_scores(name, game):
    global scores
    if name not in scores:
        scores[name] = {}
    try:
        scores[name][game] += 1
    except Exception as e:
        print(e)

In this case, print(e) gives me the game values. If I use scores[name][game] = 1 instead just to test, there is no error, but the dictionary is empty, e.g. {'John': {}, 'Alice': {}}. I can't seem to add new keys to the dictionary simply by assigning a value to that key.

How can I add to the dictionary?

Upvotes: 1

Views: 467

Answers (2)

pho
pho

Reputation: 25490

Your inner dicts need to be manager.dict() too, because they also need to be shared. This should work

def add_scores(name, game):
    global scores
    if name not in scores:
        scores[name] = manager.dict()

    scores[name][game] = scores[name].get(game, 0) + 1

I also used .get(game, 0) to get the value of game with a default of zero to get rid of the try-catch and if blocks.

Remember that once you're done, you'll need to unpack the inner nested dicts because they'll be DictProxy objects.

import copy
d = copy.deepcopy(data)
for key, val in d.items():
    d[key] = copy.deepcopy(val)
print(d)

Example (works with Intel's distribution of Python 3.6.3 on Mac):

from multiprocessing import Process, Manager
import copy

manager = Manager()
scores = manager.dict()

def add_scores(name, game):
    global scores
    if name not in scores:
        scores[name] = manager.dict()

    scores[name][game] = scores[name].get(game, 0) + 1

players = {
    "Alex": ["Soccer", "Soccer", "Tennis"], 
    "Bob": ["Tennis", "Quidditch", "Soccer", "Tennis"]
    }

for iplayer, games in players.items():
    for igame in games:
        p = Process(target=add_scores, args=(iplayer, igame))
        p.start()
        p.join()


d = copy.deepcopy(scores)
for key, val in d.items():
    d[key] = copy.deepcopy(val)

print(d)

gives the output:

{
  'Alex': {'Soccer': 2, 'Tennis': 1}, 
  'Bob': {'Tennis': 2, 'Quidditch': 1, 'Soccer': 1}
}

Upvotes: 1

pepoluan
pepoluan

Reputation: 6818

The key is to remember that Manager.dict will sync only during setitem. If you edit the value object directly, Manager.dict will not see the change and thus won't sync. (Because from Manager.dict's point of view, the key is still pointing to the same value-object; Manager.dict does not monitor changes inside the pointed value-object.)

Here's a version that runs in Python 3.9 and Python 3.6 (and every Python version in-between):

from multiprocessing import Process, Manager


def add_scores(_scores, name, game):
    # First get the inner dict
    # No need to use .setdefault() because we're going to
    # overwrite the value later, anyways
    nmgm = _scores.get(name, {})
    # Modify it
    nmgm[game] = nmgm.get(game, 0) + 1
    # Write it back so Manager.dict will sync
    _scores[name] = nmgm


players = {
    "Alex": ["Soccer", "Soccer", "Tennis"], 
    "Bob": ["Tennis", "Quidditch", "Soccer", "Tennis"]
    }


if __name__ == '__main__':
    manager = Manager()
    scores = manager.dict()

    for iplayer, games in players.items():
        for igame in games:
            p = Process(target=add_scores, args=(scores, iplayer, igame))
            p.start()
            p.join()

    print(scores)

Notice that I do not use global, rather I pass the instance of Manager.dict to the worker process. On my Python installations, initializing manager and scores outside of the __main__ structure resulted in a RuntimeError.

Result:

{'Alex': {'Soccer': 2, 'Tennis': 1}, 'Bob': {'Tennis': 2, 'Quidditch': 1, 'Soccer': 1}}

Upvotes: 3

Related Questions