CODEWITHSUNDEEP
djangorediscelerydjango-celeryperiodic-task
Marysia
Marysia

Reputation: 3

Let user create celery periodic tasks

I want to ask you about celery and redis usage in django app. I'm learning about celery for about 2 days and I'm still a bit confused :/ I've installed celery and redis and it's working just fine. My problem is, that I want user to create, update and suspend periodic tasks. I've read this article - http://kindshofer.net/2016/11/19/dynamically_adding_a_periodic_task_to_celery.html and this question - How to dynamically add / remove periodic tasks to Celery (celerybeat) a loooot of other articles, but not as useful as these two), and it appears, that I need to use django-celery. I thought, that it's a great option, but then I've read (here --> https://github.com/celery/django-celery ), that django-celery uses django result backend (it means - my db, rigth? And I want to use redis!). Is it possible to use django-celery with redis? Maybe with django-celery I don't need to use redis? Maybe there's some other way of letting user create periodic tasks? I really need my periodic tasks to be fast and lightweight, because there will be a lot of them and they need to be flawless (and that's why I thought that I need to push them somewhere else). I'd really appreciate any suggestions!

Upvotes: 0

Views: 408

Answers (2)

Rodriguez David
Rodriguez David

Reputation: 571

I think you are making some confusion... Celery needs Redit server as message broker and this 2 things must be installed globally in your system.. In order to integrate this 2 tools with django you install (the best choice would be in your virtual environment) django_celery_beat for periodic tasks with crontab and django_celery_results in order to store Celery tasks using the Django...

Of course it is possible to enable the user to launch and make customized periodic tasks.. You could (in your template, with a form) take user input and, in your views, take that input and pass it to a function on your tasks.py file.. On your tasks.py you could make a function that makes tasks!

This guide is very useful: https://github.com/codingforentrepreneurs/Guides/blob/master/all/Celery_Redis_with_Django.md

Upvotes: 0

user1600649
user1600649

Reputation:

Fast and lightweight and redis == Python RQ. However, job suspension may be an issue.

On the Celery issue: the result backend only stores results. It does not handle the queues. But even then, you can send the results to Redis:

CELERY_RESULT_BACKEND = 'django-cache'

Configure cache:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        }
    }
}

However - the reason to use the django-db backend is that it's possible to create querysets over the results. This is generally more useful, since results are displayed in views and jobs generally are not. Being able to aggregate results (like average job execution time) and create graphs, is very useful to do in Django and much easier to accomplish with the ORM then sorting / aggregating this stuff yourself in python with a key/value store.

Upvotes: 1

Related Questions