How to run the aioschedule in multiple threads

Question

import asyncio
import time
import aioschedule
from datetime import datetime


async def work_1():
    print('work1', datetime.now())
    await asyncio.sleep(30)
    
    
async def work_2():
    print('work2', datetime.now())
                

def main():    
    aioschedule.every(5).seconds.do(work_1)
    aioschedule.every(10).seconds.do(work_2)

    loop = asyncio.get_event_loop()
    while True:
        loop.run_until_complete(aioschedule.run_pending())
        time.sleep(1)
        
        
if __name__ == "__main__":
    main()

Here is the code, in theory it should run functions in parallel, i.e. work_1 and work_2 should work independently of each other. But it works in such a way that when running work_1, the script is blocked just for these 30 seconds, and only after them work_2 and work_1 start working

This is the terminal output if you run the script

work1 2024-05-28 18:12:34.051410
work2 2024-05-28 18:13:05.055715
work1 2024-05-28 18:13:09.072192

At the same time, everyone needs work1 and work2 not to wait for each other to complete, including themselves. Conditionally, work_1 should run every(5).seconds, And so it will run every 5 seconds and another 30 seconds from above inside the function

Answer 1

I don't know how helpful this will be to you. You can optionally skip to the conclusion, although the next section is the evidence for that conclusion.

The Tests

Let's take a slightly simpler case where we can better see what is happening. We will remove the call to asyncio.sleep that was in work_1 and schedule both functions for every 5 seconds:

import asyncio
import time
import aioschedule
from datetime import datetime


async def work_1():
    print('work1', datetime.now())


async def work_2():
    print('work2', datetime.now())


def main():
    print('start', datetime.now())
    aioschedule.every(5).seconds.do(work_1)
    aioschedule.every(5).seconds.do(work_2)

    loop = asyncio.get_event_loop()
    while True:
        loop.run_until_complete(aioschedule.run_pending())
        time.sleep(1)


if __name__ == "__main__":
    main()

Prints:

start 2024-05-30 06:49:48.236809
work2 2024-05-30 06:49:53.261802
work1 2024-05-30 06:49:53.262801
work2 2024-05-30 06:49:58.313681
work1 2024-05-30 06:49:58.313681
work2 2024-05-30 06:50:03.353699
work1 2024-05-30 06:50:03.353699
...

This is more or less what you would expect to see, although I would have thought that work_1 and work_2 would have run initially as soon as aioschedule.run_pending() was invoked.

So each function is to be invoked every 5 seconds. Now let's add to work_1 a call to asyncio.sleep(3):

async def work_1():
    print('work1', datetime.now())
    await asyncio.sleep(3)

work_1 is to be called every 5 seconds and takes 3 seconds to run. Since the call to await asyncio.sleep(3) should not block any other asyncio tasks from running, I would expect to see the same sort of output. What we see is:

start 2024-05-30 07:13:37.105464
work2 2024-05-30 07:13:42.155544
work1 2024-05-30 07:13:42.155544
work2 2024-05-30 07:13:47.175703
work1 2024-05-30 07:13:50.200704
work2 2024-05-30 07:13:54.209666
work1 2024-05-30 07:13:58.251530
work2 2024-05-30 07:14:02.264715
...

The first invocations of work_2 and work_1 occurs 5 seconds after the "start" message is displayed just as before. The second occurrence of work_2 occurs 5 seconds after its previous occurrence -- again, exactly as expected. However, the second occurrence of work_1 occurs 8 seconds after its previous occurrence instead of the expected 5 seconds and seems to be scheduled every 8 seconds instead of every 5 seconds.

Why should a task that is supposed to run every 5 seconds and only takes 3 seconds to execute run every 8 seconds? If a functions is to be scheduled every N seconds, then the scheduler should be noting the time a task last began and if at all possible execute the next occurrence N seconds later. It seems, however, that it is noting when the task last ended and scheduling the next occurrence N seconds after that. Moreover, the third occurrence of work_2 is now taking place approximately 7 seconds after the second occurrence and its fourth occurrence 8 seconds after its third occurrence. Why should its scheduling be affected by this change and in such an irregular fashion?.

Conclusion

I believe this module simply does not work correctly and I suggest you look elsewhere for a scheduling package. Note also that the documentation for this module is supposedly at schedule.readthedocs.io, but that references a different module named schedule that does not use asyncio. Maybe you should be looking into using that module or Python's own sched module.

Update

You could try:

import asyncio
import time
from datetime import datetime
from multiprocessing import Process


async def work_1():
    print('work1', datetime.now())
    await asyncio.sleep(30)


async def work_2():
    print('work2', datetime.now())


def _schedule(fn, frequency):
    loop = asyncio.get_event_loop()
    last_t = float('-inf')

    while True:
        t = time.time() - last_t
        if t < frequency:
            time.sleep(frequency - t)
        last_t = time.time()
        loop.run_until_complete(fn())

def schedule(fn, frequency):
    Process(target=_schedule, args=(fn, frequency)).start()

def main():
    print('start', datetime.now())
    schedule(work_1, 5)
    schedule(work_2, 10)


if __name__ == "__main__":
    main()

Prints:

start 2024-05-30 09:35:47.267258
work1 2024-05-30 09:35:47.413600
work2 2024-05-30 09:35:47.413600
work2 2024-05-30 09:35:57.419960
work2 2024-05-30 09:36:07.429956
work1 2024-05-30 09:36:17.419507
work2 2024-05-30 09:36:17.435418
work2 2024-05-30 09:36:27.443898
work2 2024-05-30 09:36:37.449400
work1 2024-05-30 09:36:47.434289
work2 2024-05-30 09:36:47.449855
work2 2024-05-30 09:36:57.464026
work2 2024-05-30 09:37:07.467038
...