CODEWITHSUNDEEP
pythondjangoapschedulerdjango-scheduler
kethan bravo
kethan bravo

Reputation: 93

What's the problem with sharing job stores in APScheduler?

I did quite get the problem that arises by sharing a job store across multiple schedulers in APScheduler.

The official documentation mentions

Job stores must never be shared between schedulers

but doesn't discuss the problems related to that, Can someone please explain it?

and also if I deploy a Django application containing APScheduler in production, will multiple job stores be created for each worker process?

Upvotes: 0

Views: 754

Answers (1)

Alex Grönholm
Alex Grönholm

Reputation: 5901

There are multiple reasons for this. In APScheduler 3.x, schedulers do not have any means to signal each other about changes happening in the job stores. When the scheduler starts, it queries the job store for jobs due for execution, processes them and then asks how long it should sleep until the next due job. If another scheduler adds a job that would be executed before that wake-up time, the other scheduler would happily sleep past that time because there is no mechanism with which it could receive a notification about the new (or updated) job.

Additionally, schedulers do not have the ability to enforce the maximum number of running instances of a job since they don't communicate with other schedulers. This can lead to conflicts when the same job is run on more than one scheduler process at the same time.

These shortcomings are addressed in the upcoming 4.x series and the ability to share job stores could be considered one of its most significant new features.

Upvotes: 1

Related Questions