CODEWITHSUNDEEP
pythonmultiprocessingshared-memory
Hazan
Hazan

Reputation: 473

Shared memory in pool of processes

I'm trying to share memory between 4 processes but I it is not working for me, I would like to get help with my problem.

Take a look at the logic of the flow:

from multiprocessing import Pool, Value
from time import sleep

def work(process):
    if process == 'A':
        sleep(secs=2)     # Processing task A
        a = True          # Marking a shared byte of completed task
    if process == 'B':
        sleep(secs=1)     # Starting processing task B
        while a is False: # Waiting until task A will complete
            pass
        sleep(secs=5)     # Continuing processing task B
        b = True          # Marking a shared byte of completed task
    if process == 'C':
        sleep(secs=7)     # Processing task C
    if process == 'D':
        sleep(secs=1)     # Starting processing task D
        while b is False: # Waiting until task B will complete
            pass
        sleep(secs=4)     # Continuing processing task D

def main():
    a = Value('b', False)
    b = Value('b', False)

    with Pool(processes=4) as pool:
        pool.map(func=work, iterable=('A', 'B', 'C', 'D'))

if __name__ == '__main__':
    main()

I can't share the values a and b, how should I do that?

Thank a lot!

Upvotes: 0

Views: 1082

Answers (2)

Booboo
Booboo

Reputation: 44313

First, function sleep does not take a keyword secs= parameter. Second, you need to pass values a and b to your function work as arguments. Third, you need to set and test the value attribute of you Value instances. Fourth, since you are passing these arguments to pool processes, you should get the Value instances from a SyncManager instance as shown below.

from multiprocessing import Pool, Manager
from time import sleep
from functools import partial

def work(a, b, process):
    if process == 'A':
        sleep(2)     # Processing task A
        print('A is marking a True')
        a.value = True          # Marking a shared byte of completed task
    elif process == 'B':
        sleep(1)     # Starting processing task B
        while a.value is False: # Waiting until task A will complete
            pass
        sleep(5)     # Continuing processing task B
        print('B is marking b True')
        b.value = True          # Marking a shared byte of completed task
    elif process == 'C':
        sleep(7)     # Processing task C
    elif process == 'D':
        sleep(1)     # Starting processing task D
        while b.value is False: # Waiting until task B will complete
            pass
        print('D has now found b to be True')
        sleep(4)     # Continuing processing task D

def main():
    manager = Manager()
    a = manager.Value('b', False)
    b = manager.Value('b', False)

    with Pool(processes=4) as pool:
        pool.map(func=partial(work, a, b), iterable=('A', 'B', 'C', 'D'))

if __name__ == '__main__':
    main()

Prints:

A is marking a True
B is marking b True
D has now found b to be True

Upvotes: 1

psykx
psykx

Reputation: 293

You can use a Queue (multiprocessing.Queue() ) to pass messages back and forth.

See this link https://pymotw.com/2/multiprocessing/communication.html

Upvotes: 0

Related Questions