CODEWITHSUNDEEP
pythonredisrabbitmqcelerymessage-queue
danish_wani
danish_wani

Reputation: 872

Using Celery with RabbitMQ as broker vs using just RabbitMQ + Pika for async tasks, advantages of using one over another

The debate I am in currently is whether we should stick with RabbitMQ implementation using Pika or move to celery, what all advantages are there if we go with Celery. From what I have understood is Celery is a distributed job queue that simplifies the management of task distribution. It uses broker (RabbitMQ, Redis and so on) for the sending and receiving the message between client and worker, it also can optionally use backend such as Redis to store the results. Where as RabbitMQ is a message Queue which can be used to perform the Jobs in async manner. Eventually if we use either RabbitMQ and implement it using Pika in python it will do the same Job which is to execute long running processes in background.

The few advantages that I see in using Celery are:

But on the other side it seems it does restrict us to use some of the features of RabbitMQ and also it has some limitations like it will connect with broker synchronously (issue on github https://github.com/celery/celery/issues/3884 ) I am familiar with this Question already asked here Why use Celery instead of RabbitMQ? but it does not seem to be clear.

Any help would be highly appreciated.

Upvotes: 3

Views: 2720

Answers (1)

DejanLekic
DejanLekic

Reputation: 19797

Erm... it seems to me like you are comparing mosquitoes and elephants here. RabbitMQ+Pika is not a replacement for Celery. However, RabbitMQ+Pika can help you implement a (miniature) service such as Celery, if that is really what you want.

If you use RabbitMQ as backend, Celery (actually kombu) will use something similar to Pika - the celery amqp project, to communicate with the broker.

Upvotes: 1

Related Questions