CODEWITHSUNDEEP
python
Sting Jia
Sting Jia

Reputation: 165

How to share global List variable between multiple processes?

When I share a global List variable between processes, I find that the variable IDs are the same (Not like string variable, the ID is different). I'm curious if the ID are the same, how each process can set the value separately? When the child process sets the value to ['main', 'beijing'], how can the parent process still keep the value ['main'] unchanged? 

#!/usr/bin/python
# -*- coding: utf-8 -*-

from multiprocessing import Pool

oTag = []
print id(oTag)
print oTag

def worker(city):
    global oTag
    oTag.append(city)
    print id(oTag)
    print oTag

def main():
    global oTag
    oTag.append("main")
    print id(oTag)
    print oTag

    cities=["beijing", "shanghai", "guangzhou", "shenzhen"]
    pool = Pool(5)
    pool.map(worker, cities)

    print id(oTag)
    print oTag

if __name__ == "__main__":
    main()

Here is the output:

4405903088
[]  
4405903088
['main']
4405903088
['main', 'beijing']
4405903088
['main', 'shanghai']
4405903088
['main', 'guangzhou']
4405903088
['main', 'shenzhen']
4405903088
['main']

Upvotes: 0

Views: 68

Answers (1)

Serge Ballesta
Serge Ballesta

Reputation: 149125

Simply because they are different processes!

C Python says that the id of a variable is the address where the variable is located (but even this is an implementation detail). But modern systems use virtual memory: the OS gives memory pages to each process and uses a mapping between virtual addresses used by the processes and the physical memory addresses. As all your processes are clones of each other (I assume fork on Linux or other Unix-like) the virtual memory addresses are the same for all the processes while each point to a distinct set of physical pages.

So every process sees its own copy of the list at the same (virtual) address but they are still different copies stored at different physical addresses.

If you want to share state between different processes using the multiprocessing module, you can use shared memory with multiprocessing.Value or multiprocessing.Array, or a multiprocessing.Manager to hold a list and multiprocessing.Proxy to access it. More details in the reference doc.

Upvotes: 1

Related Questions