CODEWITHSUNDEEP
pythonmultiprocessingpython-itertools
tipanverella
tipanverella

Reputation: 3677

using itertools with map

I am looking for the "grown-up" pythonic way to use itertools with the map function, hopefully for multiprocessing. As an example, I will use python's map function since it is simpler to use than the one in multiprocessing.

Suppose you have a function that takes say two parameters.

def twoparameterfunction(x,y):
    return somethingnice(x,y)


xs, ys = zip(*[pairs for pairs in itertools.combinations([1,2,3,4,5],2)])

results = map(twoparameterfunction,xs,ys)

is one way for me to have map apply the function to the pairs generated by itertools.combinations. Essentially, I first make a pair of list from a list of pairs, and then use that pair of list for the map function.

This can't be the way to do this! I am hoping someone will show me a better way of doing this.

Upvotes: 1

Views: 877

Answers (1)

user2357112
user2357112

Reputation: 281187

itertools.starmap provides a solution for the non-concurrent case:

from itertools import starmap

results = list(starmap(twoparameterfunction,
                       itertools.combinations([1, 2, 3, 4, 5], 2)))

For a multiprocessing map, you can't use starmap. (You also can't use multiple input iterables, so you wouldn't have been able to use the zip-transpose solution either.) In that case, it's probably best to define a helper that takes a tuple and passes the elements to twoparameterfunction:

def starfunc(x_y):
    return twoparameterfunction(*x_y)

results = some_appropriate_pool.map(
        starfunc, itertools.combinations([1, 2, 3, 4, 5], 2))

Upvotes: 2

Related Questions