CODEWITHSUNDEEP
pythonredisrabbitmqcelery
J Freebird
J Freebird

Reputation: 3910

Celery: Connect to Remote Broker to Share Tasks

I have many time-consuming tasks that need to be shared by several machines. I currently have one master machine using Celery workers to do the task. I'm using RabbitMQ as the broker and redis as the backend running in that machine locally. The master machine is also responsible for deploying tasks and return results.

I wonder if it is possible to have slave machines remotely connected to the broker and result backend in the master machine to fetch jobs, so that all the machines work together. I think I just need to configure RabbitMQ and redis settings somehow and then start the Celery workers in the slave machines. Thanks a lot.

Upvotes: 1

Views: 1496

Answers (1)

joshuad2
joshuad2

Reputation: 317

When looking at the Celery documentation there is absolutely no limitation that you can't access RabbitMQ from worker your processes as a remote server instead of just using localhost. Take a look CELERY_QUEUE_HA_POLICY here.

Upvotes: 0

Related Questions