CODEWITHSUNDEEP
pythondjangoceleryinit.d
wobbily_col
wobbily_col

Reputation: 11891

Celery, what is the recommended way to deamonise as a worker and a scheduler?

I have Django and Celery set up. I am only using one node for the worker.

I want to use use it as an asynchronous queue and as a scheduler.

I can launch the task as follows, with the -B option and it will do both.

celery worker start 127.0.0.1 --app=myapp.tasks -B

However it is unclear how to do this on production when I want to daemonise the process. Do I need to set up both the init scripts?

I have tried adding the -B option to the init.d script, but it doesn't seem to have any effect. The documentation is not very clear.

Upvotes: 2

Views: 789

Answers (2)

cmidi
cmidi

Reputation: 2010

A couple of ways to achieve this: http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html 1. Celery distribution comes with a generic init scripts located in path-to-celery/celery-3.1.10/extra/generic-init.d/celeryd this can be placed in /etc/init.d/celeryd-name and configured using a configuration file also present in the distribution which would look like the following

# Names of nodes to start (space-separated)
#CELERYD_NODES="my_application-node_1"
# Where to chdir at start. This could be the root of a virtualenv.
#CELERYD_CHDIR="/path/to/my_application"
# How to call celeryd-multi
#CELERYD_MULTI="$CELERYD_CHDIR/bin/celeryd-multi
# Extra arguments
#CELERYD_OPTS="--app=my_application.path.to.worker --time-limit=300 --concurrency=8 --loglevel=DEBUG"
# Create log/pid dirs, if they don't already exist
#CELERY_CREATE_DIRS=1

# %n will be replaced with the nodename
#CELERYD_LOG_FILE="/path/to/my_application/log/%n.log"
#CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers run as an unprivileged user
#CELERYD_USER=my_user
#CELERYD_GROUP=my_group

You can add the following celerybeat elements for celery beat configuration to the file

# Where to chdir at start.
CELERYBEAT_CHDIR="/opt/Myproject/"
# Extra arguments to celerybeat
CELERYBEAT_OPTS="--schedule=/var/run/celery/celerybeat-schedule"

This config should be then saved in (atleast for centos) /etc/default/celeryd-config-name Look at the init file for the exact location. now you can run celery as a daemon by running commands /etc/init.d/celeryd star/restart/stop

  1. Using supervisord. As mentioned in the other answer. The superviosord configuration files are also in the distribution path-to-dist/celery-version/extra/supervisord Configure using the files and use superviosrctl to run the service as a daemon

Upvotes: 0

reptilicus
reptilicus

Reputation: 10397

Personally I use Supervisord, which has some nice options and configurability. There are example supervisord config files here

Upvotes: 1

Related Questions