CODEWITHSUNDEEP
pythonmultiprocessingpool
Ram Rachum
Ram Rachum

Reputation: 88528

Python multiprocessing: Pool of custom Processes

I am subclassing the Process class, into a class I call EdgeRenderer. I want to use multiprocessing.Pool, except instead of regular Processes, I want them to be instances of my EdgeRenderer. Possible? How?

Upvotes: 8

Views: 6522

Answers (3)

mskoh52
mskoh52

Reputation: 150

This seems to work:

import multiprocessing as mp

ctx = mp.get_context()  # get the default context

class MyProcess(ctx.Process):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        print("Hi, I'm custom a process")

ctx.Process = MyProcess  # override the context's Process

def worker(x):
    print(x**2)

p = ctx.Pool(4)
nums = range(10)
p.map(worker, nums)

Upvotes: 5

Ram Rachum
Ram Rachum

Reputation: 88528

From Jesse Noller:

It is not currently supported in the API, but would not be a bad addition. I'll look at adding it to python2.7/2.6.3 3.1 this week

Upvotes: 3

David Berger
David Berger

Reputation: 12803

I don't see any hook for it in the API. You might be able to get away with replicating your desired functionality by using initializer and initargs argument. Alternately, you can build the functionality into the callable object that you use for mapping:

class EdgeRenderTask(object):
    def op1(self,*args):
        ...
    def op2(self,*args):
        ...
p = Pool(processes = 10)
e = EdgeRenderTask()
p.apply_async(e.op1,arg_list)
p.map(e.op2,arg_list)

Upvotes: 2

Related Questions