CODEWITHSUNDEEP
pythonpython-2.7python-3.xpython-multithreading
Niklas9
Niklas9

Reputation: 9386

How to make sure queue is empty before exiting main thread

I have a program that has two threads, the main thread and one additional that works on handling jobs from a FIFO queue. Something like this:

import queue
import threading

q = queue.Queue()

def _worker():
  while True:
      msg = q.get(block=True)
      print(msg)
      q.task_done()

t = threading.Thread(target=_worker)
#t.daemon = True
t.start()

q.put('asdf-1')
q.put('asdf-2')
q.put('asdf-4')
q.put('asdf-4')

What I want to accomplish is basically to make sure the queue is emptied before the main thread exits. If I set t.daemon to be True the program will exit before the queue is emptied, however if it's set to False the program will never exit. Is there some way to make sure the thread running the _worker() method clears the queue on main thread exit?

Upvotes: 4

Views: 14273

Answers (2)

Torben Klein
Torben Klein

Reputation: 3116

Put a special item (e.g. None) in the queue, that signals the worker thread to stop.

import queue
import threading

q = queue.Queue()

def _worker():
  while True:
      msg = q.get(block=True)
      if msg is None:
          return
      print(msg) # do your stuff here

t = threading.Thread(target=_worker)
#t.daemon = True
t.start()

q.put('asdf-1')
q.put('asdf-2')
q.put('asdf-4')
q.put('asdf-4')
q.put(None)
t.join()

Upvotes: 2

Harry Harrison
Harry Harrison

Reputation: 591

The comments touch on using .join(), but depending on your use case, using a join may make threading pointless.

I assume that your main thread will be doing things other than adding items to the queue - and may be shut down at any point, you just want to ensure that your queue is empty before shutting down is complete.

At the end of your main thread, you could add a simple empty check in a loop.

while not q.empty():
    sleep(1)

If you don't set t.daemon = True then the thread will never finish. Setting the thread as a daemon thread will mean that the thread does not cause your program to stay running when the main thread finishes.

Upvotes: 3

Related Questions