CODEWITHSUNDEEP
rabbitmqqueuedistributed-system
user1008636
user1008636

Reputation: 3181

Why would you run a messaging queue (eg RabbitMQ) cluster?

Overview

A RabbitMQ broker is a logical grouping of one or several Erlang nodes, each running the RabbitMQ application and sharing users, virtual hosts, queues, exchanges, bindings, and runtime parameters. Sometimes we refer to the collection of nodes as a cluster.

Why would you do this? I understand to increase durability of messages (if a node goes down, other queues still get the messages). But what about performance? How does cluster improve performance. Won't all consumers/producers connect to the master node's queue anyway? If so, aren't we still getting traffic on a single node regardless? Do we put a load balancer so traffic is directed at different nodes each time?

How does RabbitMQ cluster increase performance?

Upvotes: 0

Views: 1104

Answers (1)

theMayer
theMayer

Reputation: 16167

Well, right after that paragraph, the documentation states the following:

What is Replicated?

All data/state required for the operation of a RabbitMQ broker is replicated across all nodes. An exception to this are message queues, which by default reside on one node, though they are visible and reachable from all nodes. To replicate queues across nodes in a cluster, see the documentation on high availability (note that you will need a working cluster first).

So, you would cluster to provide higher capacity in your RabbitMQ broker than a single node can provide alone. Note that clustering by itself is not a high-availability strategy.

Your assertion that message durability is increased is false, as message queues continue to reside on one broker (unless mirroring is used).

By default, contents of a queue within a RabbitMQ cluster are located on a single node (the node on which the queue was declared) [1]

Without mirroring, when that node goes down, messages on it will be lost. The cluster will put the queue onto a different node. RabbitMQ does not handle network partitions well, so this can be a bit of a problem.

"Aren't we still getting traffic on a single node regardless?" - if you only have one queue, then yes. However, a bigger question is "why would you run a message broker with only one queue?" Similarly, if you only create queues on one node, then you will still have one point of failure in the system.

Upvotes: 0

Related Questions