CODEWITHSUNDEEP
djangocelerysupervisord
RunLoop
RunLoop

Reputation: 20396

Supervisord / Celery output logs

I am using Supervisor to manage celery. The supervisor config file for celery contains the following 2 entries:

stdout_logfile = /var/log/supervisor/celery.log
stderr_logfile = /var/log/supervisor/celery_err.log

What is confusing me is that although Celery is working properly and all tasks are being successfully completed, they are all written to celery_err.log. I thought that would only be for errors. The celery.log file only shows the normal celery startup info. Is the behaviour of writing successful task completion to the error log correct?

Note - tasks are definitely completing successfully (emails being sent, db entries made etc).

Upvotes: 4

Views: 2940

Answers (3)

Dr Manhattan
Dr Manhattan

Reputation: 14097

One issue you could be having is that python buffers output by default

In your supervisor file, you can disable this by setting the PYTHONUNBUFFERED environment variable, see below my Django example supervisor file

[program:celery-myapp]
environment=DJANGO_SETTINGS_MODULE="myapp.settings.production",PYTHONUNBUFFERED=1
command=/home/me/.virtualenvs/myapp/bin/celery -A myapp worker -B -l DEBUG
directory=/home/me/www/saleor
user=me

stdout_logfile=/home/me/www/myapp/log/supervisor-celery.log
stderr_logfile=/home/me/www/myapp/log/supervisor-celery-err.log
autostart=true
autorestart=true
startsecs=10

Upvotes: 1

GitFree
GitFree

Reputation: 41

I met the same phenomenon just like you.This is because of celery's loging mechanism. Just see the setup_task_loggers method of celery logger source.

If logfile is not specified, then sys.stderr is used.

So, clear enough? Celery uses sys.stderr when logfile is not specified.

Solution:

  1. You can use supervisord redirect_stderr = true flag to combine two log files into one. I'm using this one.
  2. Config the celery logfile option.

Upvotes: 4

Chillar Anand
Chillar Anand

Reputation: 29594

Is the behaviour of writing successful task completion to the error log correct?

No, its not. I am having same setup and logging is working fine.

celery.log has task info

[2015-07-23 11:40:07,066: INFO/MainProcess] Received task: foo[b5a6e0e8-1027-4005-b2f6-1ea032c73d34]
[2015-07-23 11:40:07,494: INFO/MainProcess] Task foo[b5a6e0e8-1027-4005-b2f6-1ea032c73d34] succeeded in 0.424549156s: 1

celery_err.log has some warnings/errors. Try restarting supervisor process.

Upvotes: 0

Related Questions