CODEWITHSUNDEEP
pythonmultiprocessing
Victor Rodriguez
Victor Rodriguez

Reputation: 39

Python Multiprocessing Module

I have this script but It returns an error. The problem is that I omit to write

if _name_ == "__main__":

This is the script I took it from a tutorial but Im adapting it to my needs

import concurrent.futures
import time

start = time.perf_counter()


def do_something(seconds):
    print(f'Sleeping {seconds} second(s)...')
    time.sleep(seconds)
    return f'Done Sleeping...{seconds}'


with concurrent.futures.ProcessPoolExecutor() as executor:
    secs = [5, 4, 3, 2, 1]
    results = executor.map(do_something, secs)

    # for result in results:
    #     print(result)

finish = time.perf_counter()

print(f'Finished in {round(finish-start, 2)} second(s)')

the problem I have is that I'm using Windows and I don't know where I have to write: if __name__ == "__main__":

Upvotes: 0

Views: 103

Answers (2)

bitranox
bitranox

Reputation: 1894

here the working, with inline comments - You need to prevent that main() is called again when forking-like behaviour under windows.

This is only necessary under windows, because windows does not support fork - therefore python simulates some "forking-like" behaviour and tries to create the same environment in the new process :

import concurrent.futures
import time

def do_something(seconds):
    print(f'Sleeping {seconds} second(s)...')
    time.sleep(seconds)
    return f'Done Sleeping...{seconds}'

def main():
    start = time.perf_counter()
    with concurrent.futures.ProcessPoolExecutor() as executor:
        secs = [5, 4, 3, 2, 1]
        results = executor.map(do_something, secs)

    # for result in results:
    #     print(result)

    finish = time.perf_counter()
    print(f'Finished in {round(finish-start, 2)} second(s)')

if __name__ == '__main__':
    # this guards main() when forking
    main()

Upvotes: 1

tdelaney
tdelaney

Reputation: 77407

Your code needs to be import safe because multiprocessing will re-import it in the subprocess. That means that you shouldn't do anything at module level that you don't want to run on mere import. For instance, creating the executor at module level means you'd have an infinite number of subprocesses created because each new one would import the module and create yet another subprocess.

Here's how you would solve the problem

import concurrent.futures
import time

def do_something(seconds):
    print(f'Sleeping {seconds} second(s)...')
    time.sleep(seconds)
    return f'Done Sleeping...{seconds}'

if __name__ == "__main__":
    start = time.perf_counter()
    with concurrent.futures.ProcessPoolExecutor() as executor:
        secs = [5, 4, 3, 2, 1]
        results = executor.map(do_something, secs)

    # for result in results:
    #     print(result)

    finish = time.perf_counter()

    print(f'Finished in {round(finish-start, 2)} second(s)')

Upvotes: 1

Related Questions