CODEWITHSUNDEEP
pythoncelery
All Іѕ Vаиітy
All Іѕ Vаиітy

Reputation: 26462

how to enable auto scaling with Celery multi workers?

The command celery worker -A proj --autoscale=10,1 --loglevel=info starts workers with auto scaling.

When creating workers with multi,

me@mypc:~/projects/x$ celery multi start myworker --autoscale=10,1
celery multi v4.1.0 (latentcall)
> Starting nodes...
    > myworker@mypc: OK

me@mypc:~/projects/kate$ celery multi show myworker
/home/me/.virtualenvs/kate/bin/python2.7 -m celery worker --detach -n myworker@mypc --pidfile=myworker.pid --logfile=myworker%I.log --executable=/home/me/.virtualenvs/x/bin/python2.7

How do I enable multi to autoscale?

Upvotes: 9

Views: 21711

Answers (1)

Lingster
Lingster

Reputation: 1087

Autoscaling will start new processes when the work load increases. In your case you have set the min number to 1, so you will only see 1 process, unless you start putting more tasks onto your queues. You can change the way the autoscaler works. See the link below for more details:

http://docs.celeryproject.org/en/latest/userguide/workers.html#autoscaling

Upvotes: 6

Related Questions