CODEWITHSUNDEEP
pythonmultiprocessing
nickponline
nickponline

Reputation: 25914

How do you share a variable between two processes in Python?

I have two processes, one adds jobs to a queue and the other takes them off the same queue and runs them. This should work as expected and I'm not sure why the worker never gets any jobs. Here's my code:

from multiprocessing import Process
from Queue import Queue
import time

q = Queue()

def queuer():
    while True:
        q.put("JOB")
        print "Adding JOB"
        time.sleep(1)

def worker():  
    while True:
        if not q.empty():
            item = q.get()
            print "Running", item
        else:
            print "No jobs"
            time.sleep(1)

a = Process(target=queuer)
a.start()

b = Process(target=worker)
b.start()

Upvotes: 3

Views: 5909

Answers (2)

wizard23
wizard23

Reputation: 301

One possibility is to use the Queue object from the multiprocessing namespace. It's described here: http://docs.python.org/2/library/multiprocessing.html#sharing-state-between-processes

So I adapted your code. I only made 2 changes: - Use multiprocessing Queue - Avoid globals and pass the queue as a parameter to the worker and queuer (this is not needed but it's good practice to keep everything tidy)

# use the Queue from the multiprocessing namespace!
from multiprocessing import Process, Queue
import time

q = Queue()

def queuer(q):
    while True:
        q.put("JOB")
        print "Adding JOB"
        time.sleep(1)

def worker(q):  
    while True:
        if not q.empty():
            item = q.get()
            print "Running", item
        else:
            print "No jobs"
            time.sleep(1)

a = Process(target=queuer, args =(q,))
a.start()

b = Process(target=worker, args = (q,))
b.start()

Upvotes: 1

Felix
Felix

Reputation: 2154

Two things:

  1. You need to pass the Queue as an argument to both processes.
  2. You should use multiprocessing.Queue, not Queue.Queue (which are for Threads)

This code works for me:

from multiprocessing import Process, Queue
import time

def queuer(q):
    while True:
        q.put("JOB")
        print "Adding JOB"
        time.sleep(1)

def worker(q):  
    while True:
        if not q.empty():
            item = q.get()
            print "Running", item
        else:
            print "No jobs"
            time.sleep(1)



if __name__ == '__main__':
    q = Queue()
    a = Process(target=queuer, args=(q,))
    b = Process(target=worker, args=(q,))
    a.start()
    b.start()

Upvotes: 9

Related Questions