CODEWITHSUNDEEP
pythonpython-3.xmultiprocessing
Kobe Janssens
Kobe Janssens

Reputation: 318

How to properly use multiprocessing in Python by using multiple files?

I have 4 files -> main.py, process1.py, process2.py, process3.py. I want to run (in a loop) all processes by using multiprocessing. But when I execute my code only 1 process works.

Code main.py:

from multiprocessing import Process
import process1, process2, process3

if __name__ == "__main__":
    p1 = Process(target=process1)
    p2 = Process(target=process2)
    p3 = Process(target=process3)
    p1.start()
    p2.start()
    p3.start()
    p1.join()
    p2.join()
    p3.join()

Code process1.py:

while True:
    print("I'm process 1")

The other processes are similar to process 1.

Output:

I'm process 1
I'm process 1
I'm process 1
I'm process 1
...

Upvotes: 3

Views: 1015

Answers (2)

n.qber
n.qber

Reputation: 384

Solution

  Transform the process1.py, process2.py and process3.py into functions

main.py should be:

from multiprocessing import Process

from process1 import process1
from process2 import process2
from process3 import process3

if __name__ == "__main__":
    p1 = Process(target=process1)
    p2 = Process(target=process2)
    p3 = Process(target=process3)
    p1.start()
    p2.start()
    p3.start()
    p1.join()
    p2.join()
    p3.join()

and process1.py should be:

def process1():
    while True:
        print("I'm process 1")

and the others, (process2 and process3) should be turned into functions too

Explanation

  Now I have realized what is happening

when you import a .py file, python executes it, so when you first

import process1, process2, process3

python starts executing the first process, so your output is not coming from line 7 at

p1.start()

but from line 2 at the imports so turning them into functions makes de import just define the process1 instead of executing it

Upvotes: 3

n.qber
n.qber

Reputation: 384

I am not sure but I think what you want to do can be done by threading module, but to do that you would need to turn the process into functions as following (process1.py, for example):

def process1():
    while True:
        print("I'm process 1")

and in the main code you can use the threading module creating a thread for each process:

from threading import Thread

from process1 import process1
from process2 import process2
from process3 import process3

if __name__ == "main":
    p1 = Thread(target=process1)
    p2 = Thread(target=process2)
    p3 = Thread(target=process3)

    p1.start()
    p2.start()
    p3.start()

if you have any process that needs arguments you can send them by:

p1 = Thread(target=process1, args=(argument1, ))

Upvotes: 0

Related Questions