CODEWITHSUNDEEP
pythonmultithreadingmultiprocessingpython-multiprocessing
Tiger Strom
Tiger Strom

Reputation: 83

Running Multiple Process At The Same Time And Returning the Results

I want to execute a function at the same time with different arguments and also have to get the return values back in my main. A pseudocode for my problem is

def Function(data):
      print("Function Running With Values" , data[0],"and",data[1],"At timestamp : " ,time.time())
      return data[0]+data[1]


Results= []
Values= [ [1,2],[2,3],[4,5],[1,4]]
#calling function for all Values and return the result in results variable in sequence.

Upvotes: 0

Views: 642

Answers (1)

Timus
Timus

Reputation: 11321

I'm not sure what you mean with "at the same time":

If a sequential processing is fine this

Results = list(map(Function, Values))
print(Results)

or more pythonic a list comprehension

Results = [Function(value) for value in Values]
print(Results)

gives you the following output

Function Running With Values 1 and 2 At timestamp :  1605276375.4642859
Function Running With Values 2 and 3 At timestamp :  1605276375.4645345
Function Running With Values 4 and 5 At timestamp :  1605276375.4647174
Function Running With Values 1 and 4 At timestamp :  1605276375.4648669
[3, 5, 9, 5]

If you actually want multiprocessing then this

import multiprocessing as mp

with mp.Pool() as p:
    Results = list(p.map(Function, Values))

print(Results)

or this

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor() as p:
    Results = list(p.map(Function, Values))

print(Results)

gives you output like

Function Running With Values 1 and 2 At timestamp :  1605276375.4532914
Function Running With Values 4 and 5 At timestamp :  1605276375.4547572
Function Running With Values 2 and 3 At timestamp :  1605276375.4549458
Function Running With Values 1 and 4 At timestamp :  1605276375.456188
[3, 5, 9, 5]

If you want multiprocessing then you should look a bit deeper into it to make sure nothing goes wrong and the processing is indeed faster. But your example is a classic MapReduce scenario that should work fine.

Is that what you were looking for?

Upvotes: 1

Related Questions