CODEWITHSUNDEEP
pythonjupyter-notebookpython-multiprocessing
Mockingbird
Mockingbird

Reputation: 115

Reaon of BrokenProcessPool error message while using concurrent.futures.ProcessPoolExecutor()?

I am trying to run this following code on Jupyter notebook (using Python 3.7.5) :

import time
import concurrent.futures

start = time.perf_counter()

def do_something(seconds):
    print(f'sleeping for {seconds} seconds')
    time.sleep(seconds)
    return 'done sleeping'

with concurrent.futures.ProcessPoolExecutor() as executor:
    results = [executor.submit(do_something, 1) for _ in range(10)]    
    for f in concurrent.futures.as_completed(results):
        print(f.result())

finish = time.perf_counter()

print(f'finished in  {round(finish-start,9)} seconds')

But, the interpreter shows an error message :

BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

I am learning multiprocessing from an YouTube tutorial where this code works fine. I haven't found any webpage discussing this error.

Upvotes: 1

Views: 671

Answers (1)

Anmol Singh Jaggi
Anmol Singh Jaggi

Reputation: 8576

The code is correct; however it will not work in Jupyter.
Its expected behaviour according to the docs:

The main module must be importable by worker subprocesses. This means that ProcessPoolExecutor will not work in the interactive interpreter.

Upvotes: 4

Related Questions