CODEWITHSUNDEEP
pythoncelerysupervisordcelerybeat
Sourabh Dev
Sourabh Dev

Reputation: 743

Celery beat using supervisor in production

I am trying to run celery beat from my virtual env using supervisor. The script doesn't seem to work

All my supervisor scripts are in the directory /etc/supervisord It has a supervisord.conf file and directory conf.d which contains the file Gorgon-celery.conf

My supervisord.conf file looks like this:

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/var/log/supervisord/main.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)
childlogdir=/var/log/supervisord            ; ('AUTO' child log dir, default $TEMP)

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[include]
files = /etc/supervisord/conf.d/*.conf

My Gorgon-celery.conf file looks like this:

[program:Gorgon-celery]
command=cd /home/ubuntu/sites/source && source ../virtualenv/bin/activate && celery -A Gorgon worker
environment=PYTHONPATH=/home/ubuntu/sites/virtualenv/bin
directory=/home/ubuntu/sites/source
numprocs=1
stdout_logfile=/var/log/celeryd/Gorgon.log
stderr_logfile=/var/log/celeryd/Gorgon.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600

[program:Gorgon-celerybeat]
command=cd /home/ubuntu/sites/source && source ../virtualenv/bin/activate && celery -A Gorgon beat --max-interval=10
environment=PYTHONPATH=/home/ubuntu/sites/virtualenv/bin
directory=/home/ubuntu/sites/source
numprocs=1
stdout_logfile=/var/log/celeryd/Gorgon-beat.log
stderr_logfile=/var/log/celeryd/Gorgon-beat.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600

Finally to the supervisor, I am using the command: sudo supervisord -c /etc/supervisord/supervisord.conf

Upvotes: 2

Views: 5020

Answers (1)

Martin Konecny
Martin Konecny

Reputation: 59601

Try replacing 

command=cd /home/sourabh_workaholic_gmail_com/sites/source && source ../virtualenv/bin/activate && celery -A Gorgon beat --max-interval=10

with 

command=bash /home/sourabh_workaholic_gmail_com/script.sh

and now in script.sh and put the following: 

#!/bin/bash    
cd /home/sourabh_workaholic_gmail_com/sites/source 
source ../virtualenv/bin/activate 
celery -A Gorgon beat --max-interval=10

I suspect you cannot use the source command with supervisorctl since it does not execute your command string in a shell. source is a command provided by the bash shell, so instead we put the commands in a bash script where they will work.

Upvotes: 4

Related Questions