Diego Jancic
Diego Jancic

Reputation: 7440

Limit number of processes in Celery with supervisor

I'm running Celery in a small instance in AWS Elastic Beanstalk.

However, when I do top, I see there are 3 celery processes running. I want to have only.

I'm running this using supervisor and in my config file I have (only showing relevant lines):

[program:celeryd]
directory=/opt/python/current/app/src
command=/opt/python/run/venv/bin/celery worker -A ..."

user=celery
numprocs=1
killasgroup=true

I've also followed the suggestion in this answer and created a file /etc/default/celeryd with this content:

# Extra arguments to celeryd
CELERYD_OPTS="--concurrency=1"

After restarting Celery (with supervisorctl -c config-file-path.conf restart celeryd), I see the 3 processes again. Any ideas? Thanks!

Upvotes: 1

Views: 848

Answers (1)

Chillar Anand
Chillar Anand

Reputation: 29534

You are starting worker with celery command. Changing /etc/default/celeryd won't have any effect on celery command. Moreover celeryd is deprecated.

When a worker is started, celery launches a default process and n(concurrency) subprocesses.

You can start the worker with

[program:celery]
command=/opt/python/run/venv/bin/celery worker -c 1 -A foo"

This will start a worker with concurrency of 1 and there will be 2 processes.

Upvotes: 1

Related Questions