I am looking for an enterprise tasks scheduler for python, like quartz is for Java. Requirements: Persistent: if the process restarts or the machine restarts, then all the jobs must stay there and must be fired after restarting. Jobs must enter and exit the scheduler in a transaction (i.e. if some database operation fail, in a database unrelated to the scheduler, then the job must not have exited or entered the scheduler). Scalability. Depends on the measure of success of the project, but I would prefer to know from the beggining that I am not starting from a dead end. Configurability: when tasks expire, how many can be fired simultaneously, etc. Any recommendations? Is there something python specific, or is it possible (or even good) to interface to Quartz from python.

pythonscheduled-tasksenterprisequartz-scheduler

Reputation: 273922

An enterprise scheduler for python (like quartz)

I am looking for an enterprise tasks scheduler for python, like quartz is for Java. Requirements:

Persistent: if the process restarts or the machine restarts, then all the jobs must stay there and must be fired after restarting.
Jobs must enter and exit the scheduler in a transaction (i.e. if some database operation fail, in a database unrelated to the scheduler, then the job must not have exited or entered the scheduler).
Scalability. Depends on the measure of success of the project, but I would prefer to know from the beggining that I am not starting from a dead end.
Configurability: when tasks expire, how many can be fired simultaneously, etc.

Any recommendations? Is there something python specific, or is it possible (or even good) to interface to Quartz from python.

Upvotes: 34

Answers (4)

kai

Reputation: 1768

This project seems not bad. a flexible python library for building your own cron-like system to schedule jobs, which is to run a tornado process to serve REST APIs and a web ui.

https://github.com/Nextdoor/ndscheduler

Upvotes: 1

inspectorG4dget

Reputation: 114035

Is APScheduler what you're looking for?

Upvotes: 28

Edward Dale

Reputation: 30143

We're using Sun Grid Engine, which is accessible through DRMAA, which happens to have Python bindings.

Upvotes: 2

user

Reputation: 5411

You can use Celery

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

Install Celery using pip install celery

Another option is using RQ.

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

Install using pip install rq.

Upvotes: 4

An enterprise scheduler for python (like quartz)

Answers (4)

Related Questions