CODEWITHSUNDEEP
rabbitmq
Reader
Reader

Reputation: 87

RabbitMQ clustering

I have created RabbitMQ cluster on single windows machine with HA policy to all and created two DISC and two RAM node and 1 STAT node. I then ran the PerfTest (rabbitmq client test utility), the result were disappointing, it was around 5000m/sec. But when I ran the same test with single RabbitMQ node it gave me good result i.e. 25000m/sec. I am unable to get what wrong is happening, its result should be impressive if run within cluster, but it is opposite. Anyone have encounter the same or if know the reason behind it. Thanks

Upvotes: 2

Views: 1098

Answers (5)

user440850
user440850

Reputation: 101

The problems is that you are running a cluster on the same machine with the same resources. The purpose of a rabbit cluster is to scale out and not scale in. In other words, to have more network connections available, more disk power of course more CPU power to handle more messages. When adding nodes on a single machine you don't scale your resources plus you are adding overheads of using a cluster. (As stated above)

Upvotes: 0

Shrey
Shrey

Reputation: 569

Just to addon to above comments.. Putting it as FYI

Upvotes: 0

joshuad2
joshuad2

Reputation: 317

As is typical with RabbitMQ, it really depends. Here are a few ways that I have found to improve performance with RabbitMQ clustering:

  • Push the messages to a set of appropriately sized memory nodes only using a load balancer
  • Keep the message size very small
  • Do not use amqp transactions or Publisher Confirms
  • Only use HA Mirrored queues for a small set of queues that you absolutely have to have the data saved
  • Set a TTL on all messages or queues using a policy

Upvotes: 0

old_sound
old_sound

Reputation: 2313

A RabbitMQ Cluster with Mirrored Queues won't go faster than a single node. Why? Clustering is there to improve reliability and fault tolerance, not to improve throughput.

What's the reason for this? When you enable mirrored queues, RabbitMQ needs to coordinate state between nodes, that is, it needs to coordinate publishes, consumers and acks, to not deliver the same message more than once, or to more than one consumer. All this coordination affects performance, but that's the tradeoff with this kind of replication.

If you need decentralised replication, then you could use the Federation Plugin

Upvotes: 4

Zoro_77
Zoro_77

Reputation: 415

The throughput rate would depend on couple of factors. In our perf tests for RabbitMQ in a cluster we observed that the rate varied depending on RabbitMQ nodes were DISC or RAM, but a big chunk of the performance variation was observed when running RabbitMQ Cluster with Mirrored Queues vs without. With Mirroring enabled we were seeing a rate of 3500 m/sec, while without it was 5000 m/sec. Also what is your message size when you run your perftest.

Upvotes: 0

Related Questions