CODEWITHSUNDEEP
pythonpython-3.xpython-multiprocessing
jalkjda asjdlka
jalkjda asjdlka

Reputation: 47

Python - using multiprocessing with multiple args doesn't work

I'm trying to use multiprocessing with multiple args to print dummy values, but this doesn't seems to work. I get error 

"f() missing 2 required positional arguments:..."

for the following code:

from multiprocessing import Pool

class tryProcessing:
    def f(self, arr, arg1, arg2):
        print(arr + " " + arg1 + " " + arg2)

    def func(self, arr, arg1, arg2):
        arg1 = "hi"
        arg2 = "hello"
        arr_a = ['1','2']
        arr_b = ['3','4','5']
        p = Pool(Processes=2)
        p.map(self.f, [[a, arg1, arg2], [b, arg1, arg2]])
        p.close

What am I doing wrong?

P.s. in this answer, he did something similar, I don't understand why his works, and mine doesn't.

Upvotes: 1

Views: 607

Answers (3)

wangolo joel
wangolo joel

Reputation: 305

There are a couples of difference with your approach compared with the solution you posted it's link there.

  • Notice positional arguments def someaction(a, b=1, c=2, d=3)
  • And you don't have positional argument def f(self, arr, arg1, arg2)

That might explain the error you are getting above. Tweaking your code this works

from multiprocessing import Pool

class tryProcessing:
    def f(self, arr):
        print(arr)

    def func(self, arr, arg1, arg2):
        arg1 = "hi"
        arg2 = "hello"
        arr_a = ['1','2']
        arr_b = ['3','4','5']
        p = Pool(2)

        data = [["a1", "b1", "c1", "d1"],
        ["a2", "b2", "c2", "d2"],
        ["a3", "b3", "c3", "d3"], ]

        p.map( self.f, data)
        p.close

t = tryProcessing()
t.func("adfasf", "dfaf", "daf")

Upvotes: 0

Mad Physicist
Mad Physicist

Reputation: 114320

You are looking for starmap, which expected the iterable to contain nested iterables of arguments to be expanded as function arguments. It uses star (splat) notation to expand, hence the name.

P.S. you never actually call p.close at the end of your function.

Upvotes: 1

Florian Bernard
Florian Bernard

Reputation: 2569

You pass one arguments, and it's an entire list.


test = map(lambda x : x, [[a, arg1, arg2], [b, arg1, arg2]]) 
print(next(test))

You can update you f func like this. 


def f(self, *args):
    arr, arg1, arg2 = args
    print(f"{arr} {arg1} {arg2}")

Upvotes: 1

Related Questions