CODEWITHSUNDEEP
pythondjangocelery
Florent
Florent

Reputation: 1928

How to properly configure Celery logging functionality with Django?

I use Django and Celery to schedule a task but I have an issue with the logger because it doesn't propagate properly. As you can see in the code below I have configured the Python logging module and the get_task_logger Celery module.

import logging
from celery import Celery
from celery.utils.log import get_task_logger

# Configure logging
logging.basicConfig(filename='example.log',level=logging.DEBUG)

# Create Celery application and Celery logger
app = Celery('capital')
logger = get_task_logger(__name__)


@app.task()
def candle_updated(d, f):

    logging.warning('This is a log')
    logger.info('This is another log')
    return d+f

I use django-celery-beat extension to setup periodic task from the Django admin. This module stores the schedule in the Django database, and presents a convenient admin interface to manage periodic tasks at runtime.

As recommended in the documentation I start the worker and the scheduler that way:

$ celery -A capital beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
celery beat v4.4.0 (cliffs) is starting.
__    -    ... __   -        _
LocalTime -> 2020-04-02 22:33:32
Configuration ->
    . broker -> redis://localhost:6379//
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> django_celery_beat.schedulers.DatabaseScheduler

    . logfile -> [stderr]@%INFO
    . maxinterval -> 5.00 seconds (5s)
[2020-04-02 22:33:32,630: INFO/MainProcess] beat: Starting...
[2020-04-02 22:33:32,631: INFO/MainProcess] Writing entries...
[2020-04-02 22:33:32,710: INFO/MainProcess] Scheduler: Sending due task Candles update (marketsdata.tasks.candle_updated)
[2020-04-02 22:33:32,729: INFO/MainProcess] Writing entries...
[2020-04-02 22:33:38,726: INFO/MainProcess] Scheduler: Sending due task Candles update (marketsdata.tasks.candle_updated)
[2020-04-02 22:33:44,751: INFO/MainProcess] Scheduler: Sending due task Candles update (marketsdata.tasks.candle_updated)

Everything seems to run fine. There is output in the console every 6 seconds (frequency of the periodic task) so it seems the task is executed in the background but I can't check it. And the problem I have is that the file example.log is empty, what could be the reason?

Upvotes: 1

Views: 2291

Answers (1)

Gabriel Desharnais
Gabriel Desharnais

Reputation: 105

Did you start a worker node as well? beat is just the scheduler, You have to run a worker as well

celery -A capital worker -l info

Upvotes: 1

Related Questions