CODEWITHSUNDEEP
pythonqueue
ByteNudger
ByteNudger

Reputation: 1561

Why is using deque of Queue.Queue considered dangerous?

In a program where I store some dict in a Queue.Queue and I have to list each entry without removing them from the queue. Now I use this code snippet to iterate over the entries:

elements = Queue.Queue()
# populate the queue

for elem in list(elements.queue):
    # print elem

In the code of Lib/Queue.py and Lib/collections.py I found that queue is a proxy to deque and in this answer of "Python: Queue.Queue vs. collections.deque" is mentioned not to use deque directly.

Is there any particular reason not to use deque? Hope someone can elaborate a little bit.

Upvotes: 0

Views: 218

Answers (1)

dano
dano

Reputation: 94871

The reason its dangerous to iterate over the internal deque of a Queue.Queue is that another thread could come along and add or remove items from the deque while you're iterating over it (by using queue.put/queue.get). This means that your iterable could change size during iteration, which causes undefined behavior in Python, and is not something you want to happen. You can iterate over the deque safely, though, by acquiring the internal mutex of the Queue first:

elements = Queue.Queue()
# populate the queue

with elements.mutex:
    for elem in list(elements.queue):
        # print elem

This way, all other threads will be prevented from adding or removing elements from the Queue while you iterate.

Now, this has nothing to do with using deque in general - which is a perfectly valid thing to do. It's just within the context of using the deque that's internal to a Queue.Queue that you need to be careful.

Upvotes: 1

Related Questions