django-celery-email task isn't executed

Question

got a bit stuck solving a problem of asynchronic email sending. I'd like to use celery, and django database as a backend. since for now the only thing I'd like to use this queue managing tool for is email, I've installed django-celery-email as well.

Following the instruction, I've made such updates to my settings file:

import djcelery
djcelery.setup_loader()

INSTALLED_APPS += ('kombu.transport.django',
                   'djcelery',
                   'djcelery_email')

BROKER_URL = 'django://'
EMAIL_BACKEND = 'djcelery_email.backends.CeleryEmailBackend'

I'm using default django SMTP with such settings:

EMAIL_USE_TLS = True
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_USER = 'cs*****@gmail.com'
EMAIL_HOST_PASSWORD = '*********'
EMAIL_PORT = 587

I did run migrate for both djcelery and komby apps. And now, using default django send_mail core method, email isn't sent.

If remove EMAIL_BACKEND option, email sending works, but horribly slow. IF not for the speed, I wouldn't look for queuing this process in a first place.

I use MySQL database as a database backend, and it's error log file didn't show anything while adding a task, so I assume, that part is fine. It seems to me, I've missed some basic installation or configuration part, that can be easily spot by an experienced celery or djcelery user, but newbie like me could've missed.

UPDATE django shell work-around:

>>> from django.core.mail import send_mail
>>> from django.conf import settings
>>> result = send_mail('test send', 'test_send_body_text', settings.EMAIL_HOST_USER,    'hornswf@gmail.com')
>>> result[0].status
u'PENDING'
>>> result[0].ready()
False
>>> result[0].failed()
False
>>> result[0].info
>>> result[0].result

after 5 minutes of waiting (without celery email sending usually takes about 10-15 seconds), I still get:

>>> result[0].status
u'PENDING'

and after 15 minutes form last check (20 all in all):

>>> result[0].status
u'PENDING'

So can anyone help me with this issue? I strongly believe it's something simple.

Sincerely yours, Sergey Aganezov.

Answer 1

I had a similar issue as noted in my comment to your question. In my case the task was sending to a different queue than the worker was listening to. By default, the worker listens to the celery queue, but can be changed with the '-Q' option. The task will by default also send to the celery queue, however I had set the 'queue' option in the CELERY_EMAIL_TASK_CONFIG to something else, not realizing this would break things. You can debug which queue your task is sending to by setting the environment variable KOMBU_LOG_DEBUG=1 and trying to send an email from manage.py shell. I deleted the 'queue' setting and emails started to work. I don't know if you are encountering the same issue but hopefully this will help in your debugging.

Answer 2

try opening a shell, try sending an email with djcelery email backend and inspect the result.

It should be a standard celery AsyncResult that gives you more info on what is happening.

to quote from the docs

results will be a list of celery AsyncResult objects that you may ignore, or use to check the status of the email delivery task, or even wait for it to complete if want. You have to enable a result backend and set ignore_result to False in CELERY_EMAIL_TASK_CONFIG if you want to use these. See the Celery docs for more info.

EDIT:

usually the PENDING state is for tasks that are waiting the execution or not known.

Task is waiting for execution or unknown. Any task id that is not known is implied to be in the pending state.

double check you have started your workers:

./manage.py celeryd -B

usually if celery can't send the task to the backend it throws an error, but the task remains pending until a worker acks it.