Can I use sleep in the main thread without blocking other threads?

Question

I am running a python script every hour and I've been using time.sleep(3600) inside of a while loop. It seems to work as needed but I am worried about it blocking new tasks. My research of this seems to be that it only blocks the current thread but I want to be 100% sure. While the hourly job shouldn't take more than 15 minutes, if it does or if it hangs, I don't want it to block the next one that starts. This is how I've done it:

import threading
import time


def long_hourly_job():
    # do some long task
    pass


if __name__ == "__main__":
    while True:
        thr = threading.Thread(target=long_hourly_job)
        thr.start()
        time.sleep(3600)

Is this sufficient?

Also, the reason I am using time.sleep for this hourly job rather than a cron job is I want to do everything in code to make dockerization cleaner.

Answer 1

I'm still testing this. I'll upload a better version later but it turns out there was an issue in my initial test.
To make this work you would have to define both functions each time you want to use it, with a new name, and put the function call you would like to happen later without blocking Blenders or other apps, interface, underneath and outside of the for loop ...
Other than that it's not too hard to get this working right off the bat... And you don't have to sit there and create a modal operator and go through all that you could just use this.

# Generator function for non-blocking sleep
def non_blocking_sleep_generator(duration):
    for _ in range(duration):
        # print("This is a repeating message.")
        yield 1.0  # Sleep for 1 second
    # function call goes here

# Function to start the generator with a specified duration
def nb_sleep(duration):
    generator = non_blocking_sleep_generator(duration)
    bpy.app.timers.register(generator.__next__)

# # Start the generator
# nb_sleep(3)

Answer 2

The code will work (i.e.: sleep does only block the calling thread), but you should be careful of some issues. Some of them have been already stated in the comments, like the possibility of time overlaps between threads.

The main issue is that your code is slowly leaking resources. After creating a thread, the OS keeps some data structures even after the thread has finished running. This is necessary, for example to keep the thread's exit status until the thread's creator requires it. The function to clear these structures (conceptually equivalent to closing a file) is called join. A thread that has finished running and is not joined is termed a 'zombie thread'. The amount of memory required by these structures is very small, and your program should run for centuries for any reasonable amount of available RAM. Nevertheless, it is a good practice to join all the threads you create.

A simple approach (if you know that 3600 s is more than enough time for the thread to finish) would be:

if __name__ == "__main__":
    while True:
        thr = threading.Thread(target=long_hourly_job)
        thr.start()
        thr.join(3600)  # wait at most 3600 s for the thread to finish
        if thr.isAlive(): # join does not return useful information
            print("Ooops: the last job did not finish on time")

A better approach if you think that it is possible that sometimes 3600 s is not enough time for the thread to finish:

if __name__ == "__main__":
    previous = []
    while True:
        thr = threading.Thread(target=long_hourly_job)
        thr.start()
        previous.append(thr)
        time.sleep(3600)
        for i in reversed(range(len(previous))):
            t = previous[i]
            t.join(0)
            if t.isAlive():
                print("Ooops: thread still running")
            else:
                print("Thread finished")
                previous.remove(t)

I know that the print statement makes no sense: use logging instead.

Answer 3

Perhaps a little late. I tested the code from other answers but my main process got stuck (perhaps I'm doing something wrong?). I then tried a different approach. It's based on threading Timer class, but trying to emulate the QtCore.QTimer() behavior and features:

import threading
import time
import traceback
    
    
class Timer:

    SNOOZE = 0
    ONEOFF = 1

    def __init__(self, timerType=SNOOZE):

        self._timerType = timerType
        self._keep = threading.Event()
        self._timerSnooze = None
        self._timerOneoff = None

    class _SnoozeTimer(threading.Timer):
        # This uses threading.Timer class, but consumes more CPU?!?!?!

        def __init__(self, event, msec, callback, *args):
            threading.Thread.__init__(self)

            self.stopped = event
            self.msec = msec
            self.callback = callback
            self.args = args

        def run(self):
            while not self.stopped.wait(self.msec):
                self.callback(*self.args)

    def start(self, msec: int, callback, *args, start_now=False) -> bool:

        started = False
        if msec > 0:
            if self._timerType == self.SNOOZE:
                if self._timerSnooze is None:
                    self._timerSnooze = self._SnoozeTimer(self._keep, msec / 1000, callback, *args)
                    self._timerSnooze.start()
                    if start_now:
                        callback(*args)
                    started = True
            else:
                if self._timerOneoff is None:
                    self._timerOneoff = threading.Timer(msec / 1000, callback, *args)
                    self._timerOneoff.start()
                    started = True
        return started

    def stop(self):
        if self._timerType == self.SNOOZE:
            self._keep.set()
            self._timerSnooze.join()
        else:
            self._timerOneoff.cancel()
            self._timerOneoff.join()

    def is_alive(self):
        if self._timerType == self.SNOOZE:
            isAlive = self._timerSnooze is not None and self._timerSnooze.is_alive() and not self._keep.is_set()
        else:
            isAlive = self._timerOneoff is not None and self._timerOneoff.is_alive()
        return isAlive
    isAlive = is_alive


KEEP = True
    
def callback():
    global KEEP
    KEEP = False
    print("ENDED", time.strftime("%M:%S"))
    
    
if __name__ == "__main__":
    count = 0
    t = Timer(timerType=Timer.ONEOFF)
    t.start(5000, callback)
    print("START", time.strftime("%M:%S"))
    while KEEP:
        if count % 10000000 == 0:
            print("STILL RUNNING")
        count += 1

Notice the while loop runs in a separate thread, and uses a callback function to invoke when the time is over (in your case, this callback function would be used to check if the long running process has finished).