CODEWITHSUNDEEP
pythonmultithreading
ForeverConfused
ForeverConfused

Reputation: 1767

Thread safe Queue

I'm using a Queue with the concurrent.futures library in python 2.7. When I run this code snippet below it's not uncommon for it to print four 1s. I want the thread queue to be a way to share data but it does not seem to be thread safe. How can I make it thread safe?

q = Queue.Queue()

def test():
    x = q.get(True)
    print x

def thread_pool():
    for x in [1,2,3,4,5]:
        q.put(x)
    #with concurrent.futures.ProcessPoolExecutor(max_workers=2) as executor:
    executor = concurrent.futures.ProcessPoolExecutor(max_workers=5)
    for x in range(5):
        executor.submit(test)

Upvotes: 0

Views: 599

Answers (1)

Kevin
Kevin

Reputation: 30151

That's not a thread pool. It's a process pool. If you want to share objects between processes, you need to use multiprocessing.Queue instead (and possibly switch your whole program over to using multiprocessing.Pool; the documentation is unclear on this point). If you want to use threads instead of processes, you need to use concurrent.futures.ThreadPoolExecutor instead of ProcessPoolExecutor.

Upvotes: 5

Related Questions