CODEWITHSUNDEEP
pythonpython-3.xmultiprocessing
Tavy
Tavy

Reputation: 871

Update DictProxy from multiple processes does not work

Why my dict does not update? Isn't DictProxy's purpose to work with cases like this ?

from multiprocessing import Pool, Manager 

def func(logtree):
    logtree['x'] = 'x'


def main():
    manager = Manager()
    logtrees = manager.dict()
    logtrees['a'] = {}
    logtrees['b'] = {}
    logtrees['c'] = {}

    pool = Pool(processes=2)
    for key in logtrees.keys():
        pool.apply_async(func, args=(logtrees[key],))

    pool.close()
    pool.join()
    print(logtrees)


if __name__ == '__main__':
    main()

actual output: {'a': {}, 'b': {}, 'c': {}}

desired output: {'a': {'x': 'x'}, 'b': {'x': 'x'}, 'c': {'x': 'x'}}

Upvotes: 2

Views: 1143

Answers (2)

Ferdinand Nittnaus
Ferdinand Nittnaus

Reputation: 33

I don't think the answer is correct. You overwrite the manager with a normal dictionary.

logtrees = {key: dict(logtrees[key]) for key in logtrees.keys()}

You can observe this by adding the following lines

from multiprocessing import Pool, Manager


def func(logtree):
    logtree['x'] = 'x'


def main():
    manager = Manager()
    logtrees = manager.dict()
    logtrees['a'] = manager.dict()
    logtrees['b'] = manager.dict()
    logtrees['c'] = manager.dict()

    pool = Pool(2)
    for key in logtrees.keys():
        pool.apply_async(func, args=(logtrees[key],))

    pool.close()
    pool.join()
    print(logtrees.__class__)  # <-------------------------------------- here
    logtrees = {key: dict(logtrees[key]) for key in logtrees.keys()}
    print(logtrees)
    print(logtrees.__class__)  # <-------------------------------------- here


if __name__ == '__main__':
    main()

Output:

<class 'multiprocessing.managers.DictProxy'>
{'a': {'x': 'x'}, 'b': {'x': 'x'}, 'c': {'x': 'x'}}
<class 'dict'>

I hope the knowledge helps

Upvotes: 1

Tavy
Tavy

Reputation: 871

Problem solved!

from multiprocessing import Pool, Manager


def func(logtree):
    logtree['x'] = 'x'


def main():
    manager = Manager()
    logtrees = manager.dict()
    logtrees['a'] = manager.dict()
    logtrees['b'] = manager.dict()
    logtrees['c'] = manager.dict()

    pool = Pool(2)
    for key in logtrees.keys():
        pool.apply_async(func, args=(logtrees[key],))

    pool.close()
    pool.join()

    logtrees = {key: dict(logtrees[key]) for key in logtrees.keys()}
    print(logtrees)


if __name__ == '__main__':
    main()

Upvotes: 0

Related Questions