CODEWITHSUNDEEP
pythonmultiprocessing
CrazyCoder
CrazyCoder

Reputation: 2378

Access a variable across different process in Python multiprocess

I'm writing some code in Python which involves multiprocessing. The code goes something like this:

from multiprocessing import Pool
global stop
stop = False

def somefunction():
    #Perform some action and then set variable stop to True which takes around 5-7 mins
    stop = True 

if __name__ == '__main__':
    p =  Process(target= somefunction)
    p.start()
    while(stop == False):
    #some other action

Explaining the above code: I need to run the while loop till the stop variable value becomes true. The value becomes True after performing some other calculation in somefunction() method in about 5-7 mins. somefunction() is triggered by a process before starting while loop.

The problem here is even though the variable is set to True using multiprocess, the while loop still running. I declared the variable as global to make it accessible in all the methods. It is still not working.

What can I try next?

Upvotes: 1

Views: 1420

Answers (2)

Booboo
Booboo

Reputation: 44013

I would just use a multiprocessing.Event instance that is passed to the subprocess as follows:

import time

def somefunction(stop):
    #Perform some action and then set Event variable stop, which takes around 5-7 mins

    time.sleep(5) # Just 5 seconds for demo purposes
    stop.set()

def do_something_else():
    print('Doing something else.')
    time.sleep(1)

if __name__ == '__main__':
    from multiprocessing import Process, Event

    stop = Event()
    p = Process(target=somefunction, args=(stop,))
    p.start()
    while(not stop.is_set()):
        do_something_else()
    p.join() # Wait for process to end

Prints:

Doing something else.
Doing something else.
Doing something else.
Doing something else.
Doing something else.

But even simpler is just testing whether the subprocess is still running if we assume that it terminates at the point in time it would have otherwise set the stop event:

import time

def somefunction():
    #Perform some action, which takes around 5-7 mins, and then terminate

    time.sleep(5) # Just 5 seconds for demo purposes

def do_something_else():
    print('Doing something else.')
    time.sleep(1)

if __name__ == '__main__':
    from multiprocessing import Process

    p = Process(target=somefunction)
    p.start()
    while(p.is_alive()):
        do_something_else()
    p.join() # Wait for process to end

Upvotes: 0

robinood
robinood

Reputation: 1248

When using multiprocessing, the processes are genereating a completely new instance of python. Therefore, a variable in a child process will not be the same as an equally named variable in the parent process. (or any other child process)

The best solution for you here is to use shared memory variables as you can see on the documentation

Upvotes: 1

Related Questions