CODEWITHSUNDEEP
pythonmultiprocessing
Peter
Peter

Reputation: 3495

Detect if main process has been quit from background process

I have a single background process running alongside the main one, where it uses Queue to communicate (using multiprocessing, not multithreading). The main process runs constantly, and the background thread runs once per queue item so that if it gets backlogged, it can still catch up. Instead of closing with the main script (I've enabled daemon for that), I would prefer it to run until the queue is empty, then save and quit.

It's started like this:

q_send = Queue()
q_recv = Queue()
p1 = Process(target=background_process, args=(q_send, q_recv))
p1.daemon = True
p1.start()

Here's how the background process currently runs:

while True:

    received_data = q_recv.get()
    #do stuff

One way I've considered is to switch the loop to run all the time, but check the size of the queue before trying to read it, and wait a few seconds if it's empty before trying again. There are a couple of problems though. The whole point is it'll run once per item, so if there are 1000 queued commands, it seems a little inefficient checking the queue size before each one. Also, there's no real limit on how long the main process can go without sending an update, so I'd have to set the timeout quite high, as opposed to instantly exiting when the connection is broken, and queue emptied. With the background thread using up to 2gb of ram, it could probably do with exiting as soon as possible.

It'd also make it look a lot more messy:

afk_time = 0
while True:

    if afk_time > 300:
        return
    if not q_recv.qsize():
        time.sleep(2)
        afk_time += 2
    else:
        received_data = q_recv.get()
        #do stuff

I came across is_alive(), and thought perhaps getting the main process from current_process() might work, but it gave a picking error when I tried to send it to the queue.

Upvotes: 2

Views: 218

Answers (1)

a_guest
a_guest

Reputation: 36329

Queue.get has a keyword argument timeout which determines the time to wait for an item if the queue is empty. If no item is available when the timeout elapses then a Empty exception is raised.

Remove and return an item from the queue. If optional args block is true and timeout is None (the default), block if necessary until an item is available. If timeout is a positive number, it blocks at most timeout seconds and raises the Empty exception if no item was available within that time. Otherwise (block is false), return an item if one is immediately available, else raise the Empty exception (timeout is ignored in that case).

So you can except that error and break out of the loop:

try:
    received_data = q_recv.get(timeout=300)
except queue.Empty:
    return

Upvotes: 1

Related Questions