CODEWITHSUNDEEP
pythonmultiprocessing
vibhud
vibhud

Reputation: 95

Unexpected behavior with multiprocessing Pool

In the code below, I was expecting the output to be 2 as I'm changing the value of config before assigning function to pool for multiprocessing, but instead I'm getting 5. I'm sure there is a good reason for it, but not sure how to explain it.

from multiprocessing import Pool 
config = 5

class Test:

  def __init__(self):
    print("This is init")

  @classmethod
  def testPrint(cls, data):
    print(config)
    print("This is testPrint")
    return config

if __name__ == "__main__" :
  pool = Pool()
  config = 2
  output = pool.map(Test.testPrint, range(10))
  print(output)

Output

5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]

Upvotes: 5

Views: 173

Answers (1)

tdelaney
tdelaney

Reputation: 77407

The new processes are created when you create the pool. After that, changes made to the parent memory space, except for stuff that is passed in a pool function like .map, are not seen by the subprocess. Forking systems like linux create copy-on-write views to the parent memory space - and that write results in a unique memory block for the parent, not seen by the subprocess. Spawning systems reimport modules (setting global variables) and then try to pickle/unpickle state for the subprocesses. In both cases, this is completed before the Pool class initialization returns to your program.

Upvotes: 5

Related Questions