CODEWITHSUNDEEP
pythonpython-multiprocessing
Rod0n
Rod0n

Reputation: 1079

Passing a list of items to multiprocessing function

I have the following function:

def create_job(target, args):
    p = multiprocessing.Process(target=target, args=args)
    p.start()
    return p

The above function gets called by:

ps = create_job(mc.collect_categories, (cat_ids),)

Being cat_ids a list of 'ids': ['ML68', 'MA453','MA33']

When I run the code I get the following error(with a list of 10 cat_ids):

TypeError: collect_categories() takes exactly 2 arguments (11 given)

I tried to pass the values with the '*' to unpack it but that wasn't the problem.

Upvotes: 7

Views: 9286

Answers (1)

unutbu
unutbu

Reputation: 879511

(cat_ids) is identical to cat_ids. Instead use (cat_ids, ) to denote a tuple with one element:

ps = create_job(mc.collect_categories, (cat_ids, ),)

Alternatively, you could define create_job using *args: 

def create_job(target, *args):
    p = multiprocessing.Process(target=target, args=args)
    p.start()
    return p

and call create_job with:

ps = create_job(mc.collect_categories, cat_ids)

Inside create_job, the variable args will be a tuple of all the positional arguments (except the first) passed to create_job -- exactly what you want to pass to Process.

Upvotes: 9

Related Questions