django-celery warnings about settings.DEBUG

Question

I have some questions regarding running djcelery.

When I run

python manage.py celeryd --setting=settings

The workers seem to have started but the warnings concern me:

C:\Python27\lib\site-packages\djcelery\loaders.py:86: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!

Googled for this warning message but no one seems to care about it. Should I be concerned?

Thanks a lot!

Answer 1

As of Django 1.8, Django no longer leaks memory when DEBUG=True.

See https://github.com/celery/celery/pull/7626. This pull request removed the out dated warning and will be included in the Celery 5.3 release.

Answer 2

Actually... Django stopped leaking memory in Django 1.8 https://github.com/django/django/commit/cfcca7ccce3dc527d16757ff6dc978e50c4a2e61

A fix for celery such that it won't complain anymore is underway.

Answer 3

The officially recommended way to start celeryd for django-celery is:

python manage.py celeryd --setting=settings

Today, I forgot this and directly start celeryd by

python -m celery.bin.celeryd --config=settings

and viola, no warnings! And it works just fine! The Celery group really need to work on their docs. Although it seems comprehensive, some key pieces are missing to cover all the grounds imho.

Answer 4

It is what it says it is - a warning that settings.DEBUG should not be set for production deployments. If you're just using it for development, then it is not something to be concerned about. In general, this ought to apply whether you use django-celery or not.

I have not looked up the details, but as mentioned in this answer:

When DEBUG is enabled Django appends every executed SQL statement to django.db.connection.queries, this will grow unbounded in a long running process environment.