CODEWITHSUNDEEP
djangocelerydjango-celerycelery-task
albinantony143
albinantony143

Reputation: 83

How to achieve below objective.?

I am using celery with Django. Redis is my broker. I am serving my Django app via Apache and WSGI. I am running celery in supervisor mode. I am starting up a celery task named run_forever from wsgi.py file of my Django project. My intention was to start a celery task when Django starts up and run it forever in the background (I don't know if it is the right way to achieve the same. I searched it but couldn't find appropriate implementation. If you have any better idea, kindly share). It is working as expected. Now due to certain issue, I have added maximum-requests-250 parameter in the virtual host of apache. By doing so when it gets 250 requests it restarts the WSGI process.

So when every time it restarts a celery task 'run_forever' is created and run in the background. Eventually, when the server gets 1000 requests WSGI process would have restarted 4 times and I end in having 4 copies of 'run_forever' task. I only want to have one copy of the task to run at any point in time. So I would like to kill all the currently running 'run_forever' task every time the Django starts.

I have tried 

from project.celery import app
from project.tasks import run_forever
app.control.purge()
run_forever.delay()

in wsgi.py to kill all the running tasks before starting `run_forever'. But didn't work

Upvotes: 0

Views: 65

Answers (1)

2ps
2ps

Reputation: 15956

I have to agree with Dave Smith here--why do you have a task that runs forever? That said, to the extent that you want to safeguard a task from running twice, there are multiple strategies you can use. The easiest for implementation is using a database entry (since databases can be transactional and if you re using django, presumably you are using at least one database). n.b., in the code snippet below, I did not put my model in the right place to be picked up by a migration--I just put it in the same snippet for ease of discussion.

import time
from myapp.celery import app
from django.db import models

class CeleryGuard(models.Model):
    task_name = models.CharField(max_length=32)
    task_id = models.CharField(max_length=32)

@app.task(bind=True)
def run_forever(self):
    created, x = CeleryGuard.objects.get_or_create(
        task_name='run_forever', defaults={
            'task_id': self.request.id
        })
    if not created:
        return
    # do whatever you want to here
    while True:
        print('I am doing nothing')
        time.sleep(1440)
    # make sure to cleanup after you are done
    CeleryGuard.objects.filter(task_name='run_forever').delete()

Upvotes: 1

Related Questions