CODEWITHSUNDEEP
pythonmultiprocessing
Ehsan
Ehsan

Reputation: 45

how to repeat a function with some arguments on some process with pool.starmap

I used multiprocessing in python for repeating a function 1000 times with one number as input on some cores and I used the code:

def GradientProccess(m):
    x=random.seed(os.getpid()*m)
    print(random.seed(os.getpid()*m))
      .
      .
      .
    return()

if __name__ == '__main__':         
    __spec__ = "ModuleSpec(name='builtins', loader=<class '_frozen_importlib.BuiltinImporter'>)"       

    with Pool(processes=8) as pool:

        result=pool.map(GradientProccess,range(1000))

But now I am going to use multiprocessing for repeating the function with 2 lists as the input of the function. I want to use starmap but how can I repeat it 1000 times? I wrote this code but it is not correct:

 def GradientProccess(m,list1,list2):
        x=random.seed(os.getpid()*m)
        print(random.seed(os.getpid()*m))
          .
          .
          .
        return()

    if __name__ == '__main__':         
        __spec__ = "ModuleSpec(name='builtins', loader=<class '_frozen_importlib.BuiltinImporter'>)"       
        list1=[4,3]
        list2=[5,7]
        with Pool(processes=8) as pool:

            result=pool.starmap(GradientProccess,[list1,list2],range(1000))

How can I use starmap for repeat a function with 2 fixed lists as arguments? Is there any other way to do this?

Upvotes: 1

Views: 284

Answers (1)

Venkatesh-Prasad Ranganath
Venkatesh-Prasad Ranganath

Reputation: 1886

Since the function consumes both lists entirely (and not a combination of elements from each list), you don't need to use starmap. Instead, you can create a partial function and use it with map as follows.

def gradiant_process(m, l1, l2):
    ...

with Pool(processes=8) as pool:
    ...
    import functools
    partial_gp = functools.partial(gradient_process, l1=list1, l2=list2)
    result = pool.map(partial_gp, range(1000))

Upvotes: 1

Related Questions