CODEWITHSUNDEEP
pythonclassmultiprocessinglocking
nima moradi
nima moradi

Reputation: 2628

python class variable won't updated in multiprocessing

I am using multiprocessing to faster a long process in python and I want to save my data in a separate class in order to make code a little cleaner but it seems that whether I change class var in the process it will roll back to the last state before the process, while in proceed it show that variable is updated.
here is simplified example

class state_mangment():
  def __init__(self):
    print('__init__')
    self.last_save = -1

  def update_state(self):
    self.last_save  =self.last_save + 1
    return self.last_save

from multiprocessing import Process, Lock
def f(l, i,persist_state ):
    l.acquire()
    try:

        print('last save is ',persist_state.update_state(),' should be ',i)
    finally:
        l.release()

if __name__ == '__main__':
    lock = Lock()
    persist_state = state_mangment()
    processes = []

    for num in range(10):
        p = Process(target=f, args=(lock, num,persist_state ))
        processes.append(p)
        p.start()

    for p in processes:
      p.join()

    print(persist_state.last_save)

here is the output, as you can see increase variable from -1 to 0 as we can see in return value but it won't start from 0 in next iteration 

 
__init__
last save is  0  should be  0
last save is  0  should be  1
last save is  0  should be  2
last save is  0  should be  3
last save is  0  should be  4
last save is  0  should be  5
last save is  0  should be  6
last save is  0  should be  7
last save is  0  should be  8
last save is  0  should be  9
-1

Upvotes: 0

Views: 688

Answers (1)

Kurtis Rader
Kurtis Rader

Reputation: 7459

There are several things wrong with your code. A function run by multiprocessing.Process() does not share the address space of the parent process. Which is why the manipulation of the persist_state object is not reflected in the parent process. That you can use a multiprocessing.Lock() object in this fashion is because that class was designed to work in that fashion when used in a multiprocessing.Process() context. That does not mean you can manipulate the state of arbitrary objects and have those manipulations reflected in the parent process.

See this description of the Manager() class for one way to solve this problem.

Upvotes: 1

Related Questions