CODEWITHSUNDEEP
pythondjangocelerydjango-celery
alacret
alacret

Reputation: 639

Celery won't discover shared_tasks in django 1.11 and celery 4.0.0 or 4.1.0

I have a layout in my project like this: (As the documentation saids it has to be)

/zonia
    /backend
        __init__.py
        celery.py
        ...
    /musics
    tasks.py
    ...
...

In the init.py:

from __future__ import absolute_import, unicode_literals

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app

__all__ = ['celery_app']

In the celery.py:

from __future__ import absolute_import, unicode_literals

import os

import environ
from celery import Celery

env = environ.Env()
environ.Env.read_env()
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('C_FORCE_ROOT', 'true')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'backend.settings')

app = Celery('backend', backend='rpc://', broker=env('broker'))

# Using a string here means the worker don't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()

@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self))

I have several shared_tasks in the tasks.py module that are like this:

@shared_task
def recalc_ranking_music():
    musics = musics_models.Music.objects.all().order_by("-num_points")
    for music, rank in enumerate(musics):
        music.ranking = rank + 1
        music.save()

When i start the celery with command: celery -A backend worker -l info

enter image description here

As you can see, the tasks that i have in the tasks.py module doesn't get read by the celery process, but the one in the celery.py folder does.

The weird thing is that i have the same exact layout in another project and it works fine.

I have two days on this, and it is really taking some time away, any help?

UPDATE: (from comment) 'musics' is a django app, so it has a __init__.py file in it.

I've also tried passing the Apps names to the auto discover method on the celery instance without any luck.

If i set in the app.autodiscover_tasks(force=True) it throws an error:

django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

Upvotes: 1

Views: 1109

Answers (3)

alacret
alacret

Reputation: 639

I found a solution, i just import the module (Django App) in the celery.py file, and it reads all the tasks.

But, this is not the behavior described in the celery documentation

Upvotes: 0

Salva Carrión
Salva Carrión

Reputation: 540

Your problem might be related to the scope in which you are running Celery (e.g.: virtual environment)

For instance, when I run celery like this:

celery -A demoproject worker --loglevel=info

It outputs just one task (the only one in my demo app):

[tasks]
  . core.tasks.demo

but when I run it from the virtualenv, it results into this:

[tasks]
  . core.tasks.demo
  . myapp.tasks.demo_task

see? It has discovered a new app just because of the environment.

Upvotes: 1

sfdye
sfdye

Reputation: 296

Does you musics directory have a __init__.py?

Also, try to specify the package name explicitly: app.autodiscover_tasks(['musics'])

Celery documentation here

Upvotes: 1

Related Questions