CODEWITHSUNDEEP
pythonpython-3.xmultiprocessing
Matthew Moisen
Matthew Moisen

Reputation: 18329

multiprocessing.Queue - Will a single thread hang on `queue.get()` if not `queue.empty()` is called before `queue.get()`?

The documentation has this to say about multiprocessing.Queue().empty():

Return True if the queue is empty, False otherwise. Because of multithreading/multiprocessing semantics, this is not reliable.

Assume I have a single thread/process that calls queue.get(), as opposed to multiple consumers.

Is it possible for the following code to hang at queue.get()?

def process_queue(queue):
    while not queue.empty():
        item = queue.get()
        print(f"The item is {item}")

I would think that so long as you have a single thread/process calling process_queue, the code will never hang.

Of course, as soon as multiple threads/processes are calling process_queue, all bets are off.

Upvotes: 1

Views: 192

Answers (1)

dano
dano

Reputation: 94961

If there is only a single consumer, you are correct, that code will never block on queue.get().

Note that your code block above would not work as expected if it happened to start running before any of your producers were able to start populating the queue, or if it happened to exhaust the queue while the producer(s) still had more items to add. That might not be relevant for your use-case, but if it is, you may want to use a sentinel/poison pill to tell your consumer when it should stop checking for new items, instead. That way it would only block on get() in cases where the producer isn't done producing new items.

Upvotes: 1

Related Questions