CODEWITHSUNDEEP
pythonmultiprocessingglobal-variables
hoan duc
hoan duc

Reputation: 70

Global variable not updated in python while using multiprocessing

I have a following simple code

from multiprocessing import Pool
x = []


def func(a):
    print(x,a)


def main():
    a = [1,2,3,4,5]
    pool = Pool(1)
    global x
    x = [1,2,3,4]
    ans = pool.map(func,a)
    print(x)

It gives me the result

[] 1
[] 2
[] 3
[] 4
[] 5
[1, 2, 3, 4]

I expected the result to reflects the change in global variable x. Which seems that the changed in global variable x is not updated before the pool call. I would like to ask what is the cause of this?

Upvotes: 3

Views: 1356

Answers (3)

hoan duc
hoan duc

Reputation: 70

So I have done what GuangshengZuo suggested, and sadly the result was not desirable. After looking deeper into it, I realized the problem was not because of script, but rather the OS.

In windows, there is no os.fork(), hence the change in global variable is not copied. But, on Unix machine, the script works fine.

Upvotes: 1

GuangshengZuo
GuangshengZuo

Reputation: 4677

I think it is because this is multiprocess, not multithread. the main process and the new process does not share a same global variable. So the new process has the copy of the main process when x is [], and after created, main process change x's value, but it does not change to new process's x.

if change the code to this :

from multiprocessing import Pool
x = []


def func(a):
    print(x,a)


def main():
    a = [1,2,3,4,5]
    global x
    x = [1,2,3,4]
    pool = Pool(1)
    ans = pool.map(func,a)
    print(x)

and the ouput will be what you want. Notice the pool = Pool(1) 's position

Upvotes: 0

pypypy
pypypy

Reputation: 1105

Two seperate processes will not share the same global variables. A multiprocessing pool abstracts away the fact that you are using two seperate processes which makes this tough to recognise.

Upvotes: 0

Related Questions