CODEWITHSUNDEEP
pythondjango
Sam Stoelinga
Sam Stoelinga

Reputation: 5021

The best way to schedule a single task at a specified time?

Let's take the following example:

We have users which asked to post the article with id:5 at 2011-04-19 20:20 So I want to create a task which will change the status of the article at 2011-04-19 20:20.

I came up with the following ideas:

  1. Implement Celery and use RabbitQM or Django-kombu: using countdown and eta: http://docs.celeryproject.org/en/v2.2.5/userguide/executing.html#eta-and-countdown
  2. Create a django management task, which will check if a article needs to be published. run this task every minute with a cron job.
  3. Create a small python application with an endless for loop, which checks every second or every 10 seconds. I will put all the articles of the current 10 minutes in memory to do faster checks if there are none it will sleep for 10 minutes.

What do you suggest in this situation? What is best for performance and scaling. Let's say every second about 100k tasks are executed.

Upvotes: 0

Views: 567

Answers (3)

Ken Cochrane
Ken Cochrane

Reputation: 77335

My recommendation would be to keep it simple to start, using a management command kicked off by CRON every minute, and when that outgrows your need switch to the distributed job approach. If you separate your code correctly, this shouldn't be that large of a change.

If you are going to be doing 100k tasks from the start, I would pick option #1 since you will be able to use celery to distribute the load over many servers. If you pick a normal cronjob setup you will need to run all of the tasks on the same server, which doesn't scale very well. Setting up RabbitMQ and maintaining it is a lot more involved then setting up a cronjob, so put that off as long as you can.

For option 2: Django-Extentions also has a cronjob like system implemented as management commands, so you wouldn't have to reinvent the wheel, and Django-Extentions has lots of other great tools, which you might want to use anyway.

http://packages.python.org/django-extensions/jobs_scheduling.html

If you go for #3 make sure you use something to keep the daemon running, If it crashes you will need to start it backup automatically. http://supervisord.org is a good choice.

Upvotes: 4

Gandi
Gandi

Reputation: 3652

Why not to give an datetime Active field, which on default will be now() method? Then create for this model a Manager which will return only active tasks, which active date will be less then now (like .get_active_only). When you would like the article to show up in future, then just save an object with a future active value. By this you will only search for active articles and skip all the upcomming ones.

Upvotes: 1

Andrew Hare
Andrew Hare

Reputation: 351526

I would definitely suggest a cron job for something like this. If you don't want to deal with the system interop you could go with something like django-cron.

Upvotes: 1

Related Questions