CODEWITHSUNDEEP
pythonmultithreadingmultiprocessingpython-multiprocessing
slant
slant

Reputation: 11

Python process does not start without a delay

I'm working on a complex application involving multiple sockets and threads/processes. For some reason, sometimes this particular process won't run unless a delay was invoked.

Something like:

class MyClass:
    def somefunc(self):
        some_thread = Thread(...)  # Some complex processing.
        some_thread.start()

        time.sleep(1)  # Some delay like this is needed. 

        some_process = Process(target=sometarget)
        print('Should start')
        some_process.start()
        print('Should have started')

    def sometarget(self):
        print('It started')

        # Do more complex processing here.

This prints out something like:

Should start
Should have started
It started  # This gets printed out only if there's a significant delay inserted

Not sure if it's just some random occurrence due to how Python schedules jobs underneath, but if I don't put a delay, it doesn't seem to run even after I wait a while.

Upvotes: 1

Views: 309

Answers (2)

Booboo
Booboo

Reputation: 44108

See my comments to your question and Tim Roberts' answer concerning the Global Interpreter Lock (GIL). I could see a scenario where the started thread ("some complex processing") had a lock on the GIL and prevented the main thread from continuing to run and get a chance to start the process-creation statements. But it would probably not have a chance to execute the delay statement (time.sleep(1)) either. And if it did, what would that accomplish? If anything, that delay would need to be in the started thread so that it would release the GIL and allow the main thread to run. But I found that it was not necessary for that delay to be inserted. I assume that even though the thread is created and "started", the main thread continues to run and gets to create the new process before the new thread actually starts execution.

This is a complete, minimal, reproducible example:

from threading import Thread
from multiprocessing import Process
import time

def doWork():
    # 100% CPU-bound
    sum = 0
    for _ in range(100_000_000):
        sum += 1

class MyClass:
    def somefunc(self):
        some_thread = Thread(target=doWork)  # Some complex processing.
        some_thread.start()

        #time.sleep(1)  # Some delay like this is needed.

        some_process = Process(target=self.sometarget) #self.sometarget is needed
        print('Should start')
        some_process.start()
        print('Should have started')

    def sometarget(self):
        print('It started')

# this is required for Windows, which I have:
if __name__ == '__main__':
    mc = MyClass()
    mc.somefunc()

Prints:

Should start
Should have started
It started

Upvotes: 0

Tim Roberts
Tim Roberts

Reputation: 54698

When you use threading, you come up against the "global interpreter lock". The Python interpreter will only allow one thread at a time to be running Python code. Until you sleep or block for I/O, your main thread will keep running, and your threads will wait. Python threading is great for I/O-bound loads, but it is nearly useless for CPU-bound loads. That's why people use multiprocessing instead, where you're actually running a separate interpreter in a separate process.

Upvotes: 1

Related Questions