CODEWITHSUNDEEP
pythonpython-3.xmultithreadingmultiprocessing
Antal Nagy
Antal Nagy

Reputation: 11

Multiprocessing in Python 3.8

I'm trying to use multiprocessing in python for the first time. I wrote a basic prime searching program and I want to run simultaneously it on each core. The problem is: when the program does the multiprocessing, it not only does the 'primesearch' function but also the beginning of the code. My expected output would be a list of prime numbers between 0 and a limit, but it writes 16 times (I have 16 cores and 16 processes) "Enter a limit: "

Here is my code:

import time
import os
from multiprocessing import Process

# Defining lists
primes = []
processes = []
l = [0]


limit = int(input('Enter a limit: '))

def primesearch(lower,upper):
    global primes
    for num in range(lower, upper):
        if num > 1:
            for i in range(2, num):
                if (num % i) == 0:
                    break
            else:
                primes.append(num)


# Start the clock
starter = time.perf_counter()

#Dividing data
step = limit // os.cpu_count()

for x in range(os.cpu_count()):
    l.append(step * (x+1))

l[-1] = limit

#Multiprocessing
for init in range(os.cpu_count()):
    processes.append(Process(target=primesearch, args=[l[init], l[init + 1],] ))

for process in processes:
    process.start()

for process in processes:
    process.join()


#End clock
finish = time.perf_counter()


print(primes)
print(f'Finished in {round(finish-starter, 2)} second')

What could be the problem?

Upvotes: 0

Views: 3959

Answers (2)

Haoming Jin
Haoming Jin

Reputation: 21

Except for the __main__ issue, your way of using primes as a global list doesn't seem to work. I imported Queue from multiprocessing and used primes = Queue() and

    size = primes.qsize()
    print([primes.get() for _ in range(size)])
    primes.close()

in the main function and primes.put(num) in your function. I don't know if it's the best way, for me this works but if N > 12000 then the console freezes. Also, in this case, using multiprocessing is actually slightly slower than single process.

If you aim for speed, you can test only to the square root of num, which saves about half of the time. There are many optimizations you can do. If you are testing huge numbers, you can use the Rabin-Miller algorithm. http://inventwithpython.com/cracking/chapter22.html

Upvotes: 0

DisappointedByUnaccountableMod
DisappointedByUnaccountableMod

Reputation: 6826

You are using Windows - If you read the Python documenation for multiprocessing, it will reveal to you that you should protect your main code using if __name__==“__main__”: This is because on Windows each process re-executes the complete main .py file.

This is used in pretty much every example in the documentation., and explained in the section at the end ‘Programming guidelines’.

See https://docs.python.org/3/library/multiprocessing.html

Upvotes: 2

Related Questions