CODEWITHSUNDEEP
pythonmultiprocessing
code base 5000
code base 5000

Reputation: 4112

Adding Event Listeners on multiprocessing.Manager

I have two machines, and right now the child is doing a while 1: statement to constantly check the BaseManager that holds the jobs. This makes it chatty. Is there a better way to do this?

Can you create an event on the manager that says "Hey I have a new job children!" like in C# where you could wire up a new event using the += and -= syntax?

Very simply the child is like this:

class QueueManager(BaseManager):
""" multiprocessing queue manager object """
     pass
if __name__ == "__main__":
    # get the get function
    QueueManager.register('get_queue')
    while 1:
       queue = m.get_queue()
       if queue.empty() == False and \
           queue.qsize() > 0:
           job_directory = queue.get()
           print job_directory

all I'm doing it pulling the job folder from the queue. I would like a better way to do this. Can events be wired up to do this?

Thank you

Upvotes: 0

Views: 568

Answers (1)

dano
dano

Reputation: 94961

There's no need for a busy loop here. Just call get_queue once, outside of the while loop, and then use queue.get() directly inside the loop:

class QueueManager(BaseManager):
""" multiprocessing queue manager object """
     pass

if __name__ == "__main__":
    QueueManager.register('get_queue')
    queue = m.get_queue()
    while 1:
       job_directory = queue.get()
       print job_directory

The manager is returning a Proxy to the Queue.Queue() object you created on the server, not a copy of the actual Queue.Queue object. Every operation you do on the Proxy gets passed through to the instance of the Queue.Queue on the server, so you can just use it like you would a normal Queue. From the client's perspective, a shared Queue instance returned by a BaseManager subclass is no different from a plain-old Queue.Queue.

Upvotes: 2

Related Questions