CODEWITHSUNDEEP
pythondjangosupervisord
Kishan Mehta
Kishan Mehta

Reputation: 2678

Supervisord configuration for Django rq not working

I am following : Putting RQ under supervisor

My job:

@job("default")
def send_mail(subject, body, sender, receivers, cc=None, bcc=None, content_type='plain', class_name=None):
    """Send email to users."""

    *code to send mail.*

When I run

python manage.py rqworker

I am able to perform tasks using rq queues. But not with supervisord configuration. Supervisord configuration:

path: /etc/supervisord/conf.d/filename.conf

[program:myworker]
; Point the command to the specific rq command you want to run.
; If you use virtualenv, be sure to point it to
; /path/to/virtualenv/bin/rq
; Also, you probably want to include a settings module to configure this
; worker.  For more info on that, see http://python-rq.org/docs/workers/

command= /home/user/.virtualenvs/my_project/bin/rq worker
process_name=%(program_name)s
stdout_logfile = /var/log/my_project/redis.log

; If you want to run more than one worker instance, increase this
numprocs=1

; This is the directory from which RQ is ran. Be sure to point this to the
; directory where your source code is importable from
directory=/home/user/github_my_projects/projects/my_project

; RQ requires the TERM signal to perform a warm shutdown. If RQ does not die
; within 10 seconds, supervisor will forcefully kill it
stopsignal=TERM

; These are up to you
autostart=true
autorestart=true

Upvotes: 4

Views: 2518

Answers (2)

Comp Guy
Comp Guy

Reputation: 71

I solved it this way.

First my supervisord.conf (place it in /etc/supervisor/). the main change from default is to put the log files in a place where I could manage them (i.e. where my other log files are)

; supervisor config file
[unix_http_server]
file=/tmp/supervisor.sock   ; the path to the socket file

[supervisord]
logfile=/home/ubuntu/backend/logs/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/home/ubuntu/backend/logs/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/home/ubuntu/backend/logs/supervisor            ; ('AUTO' child log dir, default $TEMP)


; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

[include]
files = /etc/supervisor/conf.d/*.conf

the config to run rqworks looks like this: (place in file /etc/supervisor/conf.d/rqworker.conf). for this file pay attention to the directories. I also changed the user to the same one that runs my web site:

[program:rqworker]
command= /home/ubuntu/backend/opt/api/deploy_env/bin/python manage.py rqworker
process_name=%(program_name)%(process_num)s

; If you want to run more than one worker instance, increase this
numprocs=2
user=ubuntu

; This is the directory from which RQ is ran. Be sure to point this to the
; directory where your source code is importable from
directory=/home/ubuntu/backend/opt/api/django_project/

; RQ requires the TERM signal to perform a warm shutdown. If RQ does not die
; within 10 seconds, supervisor will forcefully kill it
stopsignal=TERM

autostart=true
autorestart=true

Upvotes: 3

Kishan Mehta
Kishan Mehta

Reputation: 2678

Answering my own question.

So here is the example configuration i used in local and development servers. You can create this file using :

sudo touch /etc/supervisor/conf.d/djangorq.conf

Configuration for development server:

[program:djangorq]

command=/root/.virtualenvs/my_project/bin/rqworker
stdout_logfile=/var/log/my_project/redis.log
user=root

numprocs=1

directory=/var/www/cast-core/my_project
environment=DJANGO_CONFIGURATION=Local,DJANGO_SETTINGS_MODULE=config.local,PYTHONPATH=/var/www/projects/my_project

stopsignal=TERM

; These are up to you
autostart=true
autorestart=true

For local environment:

[program:myworker]
command= /home/user/.virtualenvs/my_project/bin/rqworker
stdout_logfile = /var/log/my_project/redis.log

numprocs=1

directory=/home/user/github_projects/projects/my_project
environment=DJANGO_CONFIGURATION=Local,DJANGO_SETTINGS_MODULE=config.local,PYTHONPATH=/home/user/github_projects/cast-core/my_project

user = root
stopsignal=TERM

autostart=true
autorestart=true

After this you start supervisor and supervisorctl

sudo service supervisor start

and then

supervisorctl reload

Then enqueue you jobs using :

send_mail.delay(#params_required_for_send_mail_method)

Upvotes: 3

Related Questions