CODEWITHSUNDEEP
pythonmultithreading
mhernandez
mhernandez

Reputation: 616

Python Queue.get(block=False) with non-empty Queue. Can it raise Queue.empty?

I'm using python's Queue.Queue to synchronize several threads. First, a Queue is populated with N elements, as in

N = 10
q = Queue.Queue()
for i in range(N):
  q.put(i)

Then, several threads consume the elements from the queue with block=False

q.get(block=False)

My question is: can the previous call to Queue.get() raise Queue.Empty in any of the first N calls?

Thanks!

Upvotes: 3

Views: 8247

Answers (1)

dano
dano

Reputation: 94961

The first N calls to get() will succeed; q.get(block=False) will only raise Queue.Empty if the queue is actually empty. All the calls to get made by your threads are synchronized, so the first N threads that get the mutex used by the Queue will successfully get an item from the Queue. If you have N+1 or more threads, any get beyond the Nth will raise Queue.Empty. This is fairly easy to see for yourself by looking at the relevant parts of the Queue code, as mentioned by Eric.

class Queue:
    """Create a queue object with a given maximum size.

    If maxsize is <= 0, the queue size is infinite.
    """
    def __init__(self, maxsize=0):
        self.maxsize = maxsize
        self._init(maxsize)
        self.mutex = _threading.Lock()
        self.not_empty = _threading.Condition(self.mutex)
        ... # Stuff we don't care about

    def get(self, block=True, timeout=None):
        self.not_empty.acquire() # Only on thread can hold this at a time
        try:
            if not block:
                if not self._qsize(): # If there's nothing in the Queue
                    raise Empty
            ... # Other stuff we don't care about, since you use block=False
            item = self._get()
            self.not_full.notify()
            return item
        finally:
            self.not_empty.release()

    def _init(self, maxsize):
        self.queue = deque()

    def _qsize(self, len=len):
        return len(self.queue)

Upvotes: 2

Related Questions