CODEWITHSUNDEEP
pythonparallel-processing
user11767946
user11767946

Reputation:

How to use parallel processing filter in Python?

I know that in Python, map is equivalent to parallel processing class of pool.map from multiprocessing module. Is there a filter equivalent in parallel programming for Python?

Upvotes: 5

Views: 2770

Answers (1)

Danilo
Danilo

Reputation: 1028

Python has most of function orientated programming paradigms that are necessary. Such as map, filter and reduce which you can find here

That being said, multiprocessing doesn't support filter because lambdas are not pickleable by default.
But you can always implement your own using functions mentioned above or reproducing them in manner that suits you using iterators, generators or list comprehension.

One of examples taken from sagemath is: 

def pool_filter(pool, func, candidates):
    return [c for c, keep in zip(candidates, pool.map(func, candidates)) if keep]

But it further depends on your implementation.

Upvotes: 4

Related Questions