CODEWITHSUNDEEP
celery
Mo J. Mughrabi
Mo J. Mughrabi

Reputation: 6997

Scaling periodic tasks in celery

We have a 10 queue setup in our celery, a large setup each queue have a group of 5 to 10 task and each queue running on dedicated machine and some on multiple machines for scaling.

On the other hand, we have a bunch of periodic tasks, running on a separate machine with single instance, and some of the periodic tasks are taking long to execute and I want to run them in 10 queues instead.

Is there a way to scale celery beat or use it purely to trigger the task on a different destination "one of the 10 queues"?

Please advise?

Upvotes: 1

Views: 636

Answers (1)

scytale
scytale

Reputation: 12641

Use celery routing to dispatch the task to where you need:

Upvotes: 1

Related Questions