CODEWITHSUNDEEP
pythondjangocelery
Brandon Taylor
Brandon Taylor

Reputation: 34593

Celery task group not being executed in background and results in exception

My Celery task isn't executing in the background in my Django 1.7/Python3 project.

# settings.py

BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULTBACKEND = BROKER_URL
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERY_ALWAYS_EAGER = False

I have celery.py in my root app module as such:

from __future__ import absolute_import

import os
import django

from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'my_app.settings')
django.setup()

app = Celery('my_app')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

and load the app in __init__.py in the root module:

from __future__ import absolute_import
from .celery import app as celery_app

My task is set up as a shared task in a tasks.py file in my app module:

from __future__ import absolute_import
from celery import shared_task

@shared_task
def update_statistics(profile, category):
    # more code

and I call the task as a group:

. . .
job = group([update_statistics(f.profile, category)
          for f in forecasts])
job.apply_async()

However, I'm not seeing any status updates in my task queue, which I am starting via:

$ celery -A my_app worker -l info

The task is being executed, just not in the background. If I add a print statement to the task code, I will see the output in my Django development server console instead of the Celery queue.

After the task runs in the foreground, I'm greeted with this exception:

'NoneType' object has no attribute 'app'

Here's the full traceback if you're interested: https://gist.github.com/alsoicode/0263d251e3744227ba46

Upvotes: 3

Views: 1547

Answers (1)

Daniel Roseman
Daniel Roseman

Reputation: 600041

You're calling the tasks directly in your list comprehension when you create the group, so they're executed then and there. You need to use the .subtask() method (or its shortcut, .s()) to create the subtasks without calling them:

job = group([update_statistics.s(f.profile, category) for f in forecasts])

Upvotes: 5

Related Questions