CODEWITHSUNDEEP
pythonpython-3.7python-multiprocessing
Lijo Jose
Lijo Jose

Reputation: 337

Multiprocessing Manager Dict is not getting updated

I was trying to capture start and end time of each process by passing a Manager.dict. When I am using fun its not working as expected, but for fun_1 its working. I am running this code on python 3.7 (IDE VS Code)

import time
from multiprocessing import Manager
from multiprocessing.pool import Pool


def fun(a, d):
    if a not in d:
        d[a] = list()

    d[a].append(time.time())
    time.sleep(1)
    d[a].append(time.time())


def fun_1(a, d):

    s = time.time()
    time.sleep(1)
    e = time.time()
    d[a] = [s, e]


if __name__ == '__main__':
    pool = Pool(4)
    m = Manager()
    d = m.dict()
    pool.starmap(fun, [(i, d) for i in range(10)])
    print(d) # {3: [], 0: [], 2: [], 1: [], 4: [], 5: [], 6: [], 7: [], 8: [], 9: []}

Looks like I am missing something. Thanks in Advance

Upvotes: 0

Views: 1086

Answers (1)

Marc
Marc

Reputation: 1629

This thread explains your problem: https://bugs.python.org/issue36119

As a way to solve this you should either perform a __setitem__ operation allowing the system to notice a change in the object which propagates to the shared manager object. This is why your fun_1 does work as your perform d[a] = [s,e].

Another option is that you could predefine manager.list() objects instead of normal list objects which should perform the way you expect them to in this situation.

Upvotes: 1

Related Questions