CODEWITHSUNDEEP
pythonmultiprocessingqueue
Rafe
Rafe

Reputation: 558

python multiprocessing and queue access issues

I've got the following code: 

import multiprocessing
import queue
import time

to_delete = queue.Queue()

def parallel(args):
    return para_print(*args)

def para_print(sngl, dbl, iter):
    to_delete.put(f"{sngl}, {dbl}, {iter}")

if __name__ == '__main__':
    multiprocessing.freeze_support()

    expression_list = [('a', 'aa', 1), ('b', 'bb', 2), ('c', 'cc', 3), ('d', 'dd', 4)]
    pool = multiprocessing.Pool(multiprocessing.cpu_count() - 1)
    result = pool.map(parallel, expression_list)

    print(to_delete.qsize())
    while not to_delete.empty():
        print(to_delete.get())

The result is '0' printed as the queue size, with nothing getting put correctly into the queue - or pulled from it. What on earth am I doing wrong here?

Upvotes: 0

Views: 65

Answers (1)

bivouac0
bivouac0

Reputation: 2570

You should be using the queue from multiprocessing. The standard one doesn't work correctly between processes. Here's the revised code..

import multiprocessing

to_delete = multiprocessing.Queue()

def parallel(args):
    return para_print(*args)

def para_print(sngl, dbl, iter):
    to_delete.put(f"{sngl}, {dbl}, {iter}")


if __name__ == '__main__':
    multiprocessing.freeze_support()

    expression_list = [('a', 'aa', 1), ('b', 'bb', 2), ('c', 'cc', 3), ('d', 'dd', 4)]
    pool = multiprocessing.Pool(multiprocessing.cpu_count() - 1)
    result = pool.map(parallel, expression_list)

    print(to_delete.qsize())
    while not to_delete.empty():
        print(to_delete.get())

Upvotes: 1

Related Questions