CODEWITHSUNDEEP
pythonscheduled-tasksschedulingapscheduler
J.Wang
J.Wang

Reputation: 1241

How to add jobs to threadpool with APScheduler

I am new to APScheduler
For testing I have set the max_worker=2 and added 4 different jobs to scheduler. I intended to run 2 jobs in parallel because of the limitation of threadpool, I thought that after finishing certain job another added job will threw into threadpool, but the result below is not what I expected, it just kept running first 2 jobs. 

Arg is 1, time is Mon Apr 18 19:51:33 2016
Arg is 2, time is Mon Apr 18 19:51:33 2016
No handlers could be found for logger "apscheduler.scheduler"
Arg is 1, time is Mon Apr 18 19:51:37 2016  
Arg is 2, time is Mon Apr 18 19:51:37 2016
Arg is 1, time is Mon Apr 18 19:51:41 2016
Arg is 2, time is Mon Apr 18 19:51:41 2016
Arg is 1, time is Mon Apr 18 19:51:45 2016
Arg is 2, time is Mon Apr 18 19:51:45 2016
...

And here is the code: 

import time
from apscheduler.schedulers.blocking import BlockingScheduler

def job(arg):
    print 'Arg is %s, time is %s' % (arg, time.ctime())
    time.sleep(2)

if __name__ == '__main__':
    scheduler = BlockingScheduler({'apscheduler.executors.default': {'class': 'apscheduler.executors.pool:ThreadPoolExecutor', 'max_workers': '2'}})
    scheduler.add_job(job, 'interval', seconds=2, args=(1,))
    scheduler.add_job(job, 'interval', seconds=2, args=(2,))
    scheduler.add_job(job, 'interval', seconds=2, args=(3,))
    scheduler.add_job(job, 'interval', seconds=2, args=(4,))

    try:
        scheduler.start()
    except (KeyboardInterrupt, SystemExit):
        pass

Any ways to add remaining jobs to threadpool or something like queue scheme or multiprocessing.dummy.pool

Upvotes: 0

Views: 2410

Answers (2)

Alex Grönholm
Alex Grönholm

Reputation: 5911

Perhaps you'd like to use more than one thread pool executor and limit certain jobs to certain executors?

Upvotes: 0

alpo
alpo

Reputation: 151

Jobs which miss scheduled time by any reason are recognized as misfired and these jobs are not launched. In your case jobs are misfired because of lack of free worker. Add parameter misfire_grace_time=1 (or with another time) to the add_job call.

Upvotes: 2

Related Questions