CODEWITHSUNDEEP
pythonconcurrent.futures
Retko
Retko

Reputation: 382

Make concurrent.futures.ProcessPoolExecutor skip all errors / exceptions, but continue

here is my simple example where I run a function using ProcessPoolExecutor, passing list o numbers that will be iterated over. Another argument (b) is a constant in each iteration (that's why _helper function).

I put some value in a list, which will cause an exception to occur - 'a'. I want to make the processing to continue, skipping this part. Unfortunately, it is stopping on this value:

How can I make it continue? printing: 2 3 5 6

Thanks

import concurrent.futures as cf

workers = 4

def f(a, b):
    return a + b

def _helper(x):
    return f(x, 1)

my_iterable_collection = [1,2,'a',4,5]

def main():
    with cf.ProcessPoolExecutor(max_workers=workers) as executor:
        try:
            for result in executor.map(_helper, my_iterable_collection):
                print(result)
        except Exception:
            pass


if __name__ == '__main__':
    main()

# 2
# 3

Upvotes: 1

Views: 701

Answers (1)

Varist
Varist

Reputation: 46

You can try this way:

def f(a, b):
    return a + b

def _helper(x):
    try:
        return f(x, 1)
    except Exception:
        pass

my_iterable_collection = [1,2,'a',4,5]

def main():
    with cf.ProcessPoolExecutor(max_workers=workers) as executor:
        for result in executor.map(_helper, my_iterable_collection):
            if result:
                print(result)

Upvotes: 3

Related Questions