CODEWITHSUNDEEP
pythonmultiprocessing
DPM
DPM

Reputation: 935

Multiprocessing does not output anything

I am trying to create a pool inside a pool to parallelize a for cycle. I'm trying to do this to see if it is faster than running a for cycle with only one pool creation. My issue is that the code I wrote doesn't seem to ever finish running and I don't quite get why. Here is the code:

import numpy as np
import multiprocessing as mp
import time
cpus = mp.cpu_count() - 1

def f(x):
    lista = list(pool.map(time.sleep, [1,2,3] * x))
    print('done')
    return lista

pool = mp.Pool(cpus)
lista2 = pool.map(f, range(2))
pool.close()
pool.join()

Upvotes: 2

Views: 98

Answers (1)

Michael
Michael

Reputation: 2414

From the docs: "Note that the methods of a pool should only ever be used by the process which created it."

https://docs.python.org/3.4/library/multiprocessing.html#module-multiprocessing.pool

Also note that the processes started by Pool are daemonic processes that are not allowed to have their own children. This may explain why you're experiencing a deadlock. According to this blog post you should have seen an exception getting raised over this, I'm not sure why you're not experiencing that: https://blog.mbedded.ninja/programming/languages/python/python-multiprocessing/

Upvotes: 2

Related Questions