CODEWITHSUNDEEP
pythongevent
onlytiancai
onlytiancai

Reputation: 168

Why use threading data race will occur, but will not use gevent

My test code is as follows, using threading, count is not 5,000,000 , so there has been data race, but using gevent, count is 5,000,000, there was no data race .

Is not gevent coroutine execution will atom "count + = 1", rather than split into a one CPU instruction to execute?

# -*- coding: utf-8 -*-
import threading

use_gevent = True        
use_debug = False        
cycles_count = 100*10000


if use_gevent:
    from gevent import monkey
    monkey.patch_thread()

count = 0


class Counter(threading.Thread):
    def __init__(self, name):
        self.thread_name = name
        super(Counter, self).__init__(name=name)

    def run(self):
        global count
        for i in xrange(cycles_count):
            if use_debug:
                print '%s:%s' % (self.thread_name, count)
            count = count + 1

counters = [Counter('thread:%s' % i) for i in range(5)]
for counter in counters:
    counter.start()
for counter in counters:
    counter.join()

print 'count=%s' % count

Upvotes: 5

Views: 790

Answers (1)

Leonardo.Z
Leonardo.Z

Reputation: 9801

compound assignments(+=) are not atomic in Python, gevent does not change that.

The count with using gevent patched thread is always 5,000,000 because gevent coroutines are not system threads, the context-switch of gevent is not controlled by the os, and gevent will not switch until a thread is blocked(io happens) .

For normal threads, things are different. compound assignments contains 3 steps( a. read the value, b. increase the value, c. assign the changed value), a thread switch could happen between any these steps.

Check the code below, I add a new thread which will run a infinite loop. For normal threads, count can be printed, because os will switch threads automatically. But for gevent threads, count will not be printed, because once the Infinite thread is executed, it never ends and gevent will not switch to other threads because no IO occurs.

# -*- coding: utf-8 -*-
import threading

use_gevent = True        
use_debug = False        
cycles_count = 100
if use_gevent:
    from gevent import monkey
    monkey.patch_thread()

count = 0

class Counter(threading.Thread):
    def __init__(self, name):
        self.thread_name = name
        super(Counter, self).__init__(name=name)

    def run(self):
        global count
        for i in xrange(cycles_count):
            if use_debug:
                print '%s:%s' % (self.thread_name, count)
            if use_gevent:
                print self.thread_name
            count = count + 1

class Infinite(threading.Thread):
    def run(self):
        if use_gevent:
            print 'Infinite ...'
        while True:
            pass


counters = [Counter('thread:%s' % i) for i in range(5)]
for counter in counters:
    counter.start()
infinite = Infinite()
infinite.start()
for counter in counters:
    counter.join()

print 'count=%s' % count

Upvotes: 5

Related Questions