CODEWITHSUNDEEP
pythonmultithreading
avasin
avasin

Reputation: 9726

Why global var is not incremented when using threads in python?

For my code, pytest_status['finished'] variable is incrementing very interesting way. Sometimes i can see '2' or '3' (I expect corrent incrementing for every run_test function call):

test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 1/53: success
test 2/53: success
test 1/53: success
test 1/53: success

and so on. I specially moved it out of lock, my code:

import subprocess
from multiprocessing import Pool, Lock

pytest_status = {
    'finished' : 0,
    'total'    : 0
}

print_lock = Lock()

def run_test(test):
    status = subprocess.call(...)

    global pytest_status
    pytest_status['finished'] += 1

    print_lock.acquire()
    print 'test ' + str(pytest_status['finished']) + '/' + str(pytest_status['total']) + ': ' + ('success' if status == 0 else 'failure')    
    print_lock.release()

def main():
    params = [...]

    global pytest_status    
    pytest_status['total'] = len(params)

    print 'will perform ' + str(pytest_status['total']) + ' tests'

    pool = Pool(30)
    pool.map(run_test, params)
    pool.close()
    pool.join()

if __name__ == '__main__':
    main()

Upvotes: 0

Views: 46

Answers (2)

Darth Kotik
Darth Kotik

Reputation: 2351

You can use Pipe or Query if you want to transfer data between processes.

Upvotes: 1

Daniel Roseman
Daniel Roseman

Reputation: 599600

You're not using threads. You're using processes. By definition, these have separate copies of all data. The only time you'd get 2 or 3 is when the same process happened to run more than one of the jobs.

Upvotes: 3

Related Questions