CODEWITHSUNDEEP
pythonmultiprocessingglobal-variables
Wawel100
Wawel100

Reputation: 1271

Python multiprocessing: Global name is not defined

I am trying to use the Python multiprocessing lib but run into problems with globally defined variables. For some reason I get the error "NameError: global name ... is not defined" The original program is quite involved so I have reproduced my problem in the code below: 

import multiprocessing
import time
import timeit

def setGlobal():

    global globalVar
    globalVar = 2

def mp_worker(arguments):

    # do some processing.... 
    data = arguments[0]    
    # need to use global data
    the_time = globalVar    

    time.sleep(int(the_time))

def f(processes = 3):

    global pool   
    pool = multiprocessing.Pool(processes)

def mp_handler(number):

    inputs = []
    inputs.append([1]) 
    inputs.append([2]) 

    print("Begin multiprocessing test...for test "+str(number))
    start_time = timeit.default_timer()    
    pool.map(mp_worker, inputs)
    print("Time taken to calculate trades: %4.4f " %(timeit.default_timer() - start_time))


if __name__ == '__main__':
    #define global variable
    setGlobal()
    # initialise pool
    f(3)

    # Get error: NameError: global name 'globalVar' is not defined 
    mp_handler(1)

Now a potential solution I tried was: 

import multiprocessing
import time
import timeit

globalVar = 4

def setGlobal(number):

    global globalVar
    globalVar = number

def mp_worker(arguments):

    # do some processing.... 
    data = arguments[0]    
    # need to use global data
    print("GlobalVar in my_worker = " + str(globalVar))
    the_time = globalVar
    time.sleep(int(the_time))

def f(processes = 3):

    global pool   
    pool = multiprocessing.Pool(processes)

def mp_handler(number):

    inputs = []
    inputs.append([1]) 
    inputs.append([2]) 
    print("GlobalVar in my_handler = " + str(globalVar))

    print("Begin multiprocessing test...for test "+str(number))
    start_time = timeit.default_timer()    
    pool.map(mp_worker, inputs)
    print("Time taken to calculate trades: %4.4f " %(timeit.default_timer() - start_time))


if __name__ == '__main__':

    # initialise pool
    f(3)

    mp_handler(1)

    # re-set global variable... has no effect   
    setGlobal(1)
    mp_handler(2)

However when I call setGlobal(1) it has no effect in mp_worker? The globally defined value is 4 when I need it to be 1. Why is this and can this problem be overcome?

Upvotes: 2

Views: 5787

Answers (1)

Jonathan
Jonathan

Reputation: 284

The global statement does not make a variable global, but rather makes the variable accessible within the scope of a function.

Simply setting the variable at the top level (i.e. outside any function or class) makes it intrinsically global. So set the value outside any functions and classes, then declare it using global var_name when you need to change it within a function. Note that you do not need to use the global command to read a global variable, only if you need to change it within the function.

Be aware that global variables do not play nicely between multiprocessing processes: I have found it necessary to use several multiprocessing.Queue objects to send values between processes (sadly).

Upvotes: 1

Related Questions