CODEWITHSUNDEEP
pythonmultiprocessingthreadpool
cts
cts

Reputation: 1820

how do I use key word arguments with python multiprocessing pool apply_async

I'm trying to get to grips with pythons multiprocessing module, specifically the apply_async method of Pool. I'm trying to call a function with arguments and keyword arguments. If I call the function without kwargs it's fine but when I try to add in a keyword argument I get: TypeError: apply_async() got an unexpected keyword argument 'arg2' Below is the test code that I'm running

#!/usr/bin/env python
import multiprocessing
from time import sleep
def test(arg1, arg2=1, arg3=2):
    sleep(5)

if __name__ == '__main__':
    pool = multiprocessing.Pool()
    for t in range(1000):
        pool.apply_async(test, t, arg2=5)
    pool.close()
    pool.join()

How can I call the function so that it accepts keyword arguments?

Upvotes: 23

Views: 14763

Answers (3)

Chang
Chang

Reputation: 71

original answer: python multiprocessing with boolean and multiple arguments

apply_async has args and kwds keyword arguments which you could use like this:

res = p.apply_async(testFunc, args=(2, 4), kwds={'calcY': False})

Upvotes: 4

artoonie
artoonie

Reputation: 847

Janne's answer didn't work for me in python 2.7.11 (not sure why). The function test() was receiving the key (arg2), not the value (5).

I fixed this by creating a wrapper around test:

def test2(argsDict):
    test(**argsDict)

Then calling

pool.apply_async(test2, (t,), [dict(arg2=5)])

Upvotes: 1

Janne Karila
Janne Karila

Reputation: 25197

Pass the keyword args in a dictionary (and the positional arguments in a tuple):

pool.apply_async(test, (t,), dict(arg2=5))

Upvotes: 29

Related Questions