CODEWITHSUNDEEP
pythonrediscelerydjango-celery
Matt
Matt

Reputation: 5168

Celery task does not timeout

I would like to verify that setting time limits for celery tasks work.

I currently have my configuration looking like this:

CELERYD_TASK_SOFT_TIME_LIMIT = 30
CELERYD_TASK_TIME_LIMIT = 120

task_soft_time_limit = 29
task_time_limit = 44_LIMIT = 120

I am overloading the timeout parameters because it appears that there name change coming and I just want to be sure that I hit at least one timeout.

But when I run a test in the debugger the cellery app.conf dictionary looks like this:

(Pdb) app.conf['task_time_limit'] == None
True
(Pdb) app.conf['task_soft_time_limit'] == None
True
(Pdb) app.conf['CELERYD_TASK_SOFT_TIME_LIMIT']
30
(Pdb) app.conf['CELERYD_TASK_TIME_LIMIT']
120

I've written a test which I believe would trigger the timeout but no error is ever raised:

@app.task(soft_time_limit=15)
def time_out_task():
    import time
    count = 0
    #import pdb; pdb.set_trace()
    while count < 1000:
        time.sleep(1)
        count += 1
        print(count)

My questions are as follows:

  1. What are the conical settings I should set for a soft and hard time limit?
  2. How could I execute a task in a test which proves to me that the time limits are in place.

Thanks

Upvotes: 0

Views: 5859

Answers (2)

ibuler
ibuler

Reputation: 13

In django settings CELERY_TASK_TIME_LIMIT is working for me

CELERY_TASK_TIME_LIMIT = 60

Upvotes: 0

Matt
Matt

Reputation: 5168

I solved the issue by changing the way I was testing and by changing the way I was importing the Celery configuration.

Initially, I was setting the configuration by importing a Django settings object:

app = Celery('groot')
app.config_from_object('django.conf:settings', namespace='CELERY')

But this was ignoring the settings with the CELERYD_... prefix. Thus I used the new notation and called the following method:

app.conf.update(
    task_soft_time_limit=30,
    task_time_limit=120,
)

I also changed from testing this in the Django test environment to spinning up an actual Celery worker and sending the task to the worker. If someone would supply a solution for how to test the timeout settings in the unit test it would be much appreciated.

Upvotes: 1

Related Questions