CODEWITHSUNDEEP
djangocelerysupervisordcelery-taskcelerybeat
Hassan Baig
Hassan Baig

Reputation: 15814

Celery beat serving old (removed) task

Under supervisor, celery beat serves periodic tasks to celery workers for a Django app of mine. I have 4 tasks, task1, task2, task3, and task4. Recently I made a 5th task: task5.

My problem is that I commented out task5 from my workers, removed its mention from settings.py and restarted celerybeat and my celery workers. But I still see task5 periodically showing up (throwing an error in the workers' logs naturally).

Why is this happening, and how can I update the periodic tasks?

In settings.py, I have:

    import djcelery
    djcelery.setup_loader()
    # config settings for Celery Daemon

    # Redis broker
    BROKER_URL = 'redis://localhost:6379/0'

    BROKER_TRANSPORT = 'redis'

    # List of modules to import when celery starts, in myapp.tasks form. 
    CELERY_IMPORTS = ('myapp.tasks', )  

    CELERY_ALWAYS_EAGER = False

    CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
    #The backend is the resource which returns the results of a completed task from Celery. 6379 is the default port to the redis server.

    CELERY_ACCEPT_CONTENT = ['json']
    CELERY_TASK_SERIALIZER = 'json'
    CELERY_RESULT_SERIALIZER = 'json'
    CELERY_IGNORE_RESULT=True

    from datetime import timedelta

    CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'

    CELERYBEAT_SCHEDULE = {
        'tasks.task1': {
            'task': 'tasks.task1',
            'schedule': timedelta(seconds=45),
        },
        'tasks.task2': {
            'task': 'tasks.task2',
            'schedule': timedelta(seconds=60),  # execute every 60 seconds
            'args': (),
        },
        'tasks.task3': {
            'task': 'tasks.task3',
            'schedule': timedelta(seconds=90),  # execute every 90 seconds
            'args': (),
        },
        'tasks.task4': {
            'task': 'tasks.task4',
            'schedule': timedelta(seconds=90),  # execute every 90 seconds
            'args': (),
        },
    }

/etc/supervisor/conf.d/celerybeat.conf contains the following:

command=python manage.py celery beat -l info
directory = /home/myuser/myproject/
environment=PATH="/home/myuser/envs/myenv/bin",VIRTUAL_ENV="/home/myuser/envs/myenv",PYTHONPATH="/home/myuser/envs/myenv/lib/python2.7:/home/myuser/envs/myenv/lib/python2.7/site-packages"
user=mhb11
numprocs=1
stdout_logfile = /etc/supervisor/logs/celerybeat.log
stderr_logfile = /etc/supervisor/logs/celerybeat.log
autostart = true
autorestart = true
startsecs=10
stopwaitsecs = 600
killasgroup=true
priority=999

Ask me for more info if you need it. Thanks in advance.

Upvotes: 5

Views: 4771

Answers (2)

Madalosso
Madalosso

Reputation: 126

Another possible reason is that as you're using

CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'

your tasks are being stored on your database and they won't be deleted just by removing it from CELERYBEAT_SCHEDULE.

You need to delete or disable unwanted periodic tasks (easily done through djcelery app on your admin, if you have it enabled)

https://docs.celeryproject.org/en/stable/userguide/configuration.html#std:setting-beat_scheduler

Upvotes: 7

creativeChips
creativeChips

Reputation: 1197

I believe you already have tasks pending in the queue.

You will need to purge them. Or, if you know their id's (note that unless you overrode it, task ids are uniquely generated .. these are not the name of the task), you can also revoke them.

Upvotes: 4

Related Questions