CODEWITHSUNDEEP
rabbitmqload-balancinghigh-availability
GGarcia
GGarcia

Reputation: 1

Rabbitmq federated queue with mirror

I am new to RabbitMQ and not too sure if my goal is achievable. Basically, my goal is to be able to setup a highly available and load balanced queues.

So given 3 nodes with 1 queue each (let's say Node1 has Queue1, Queue2, Queue3. Node2 also has Queue1, Queue2, Queue3, and the same for Node3). Each queue has it's own consumer.

I would like to load balance the messages in the queues so that producers can send the message to any of the queue and get the consumer to process the queue.

I would also like to implement mirroring so that if one of the Nodes goes down (lets say Node1 goes down for some reason), the messages sent to queues in Node1 will not be lost and will still be processed.

From my understanding in Clustering, messages are stored to where the master queue is so therefore I didn't achieved the load balanced queue that I want.

Is my goal achievable? If yes, how can I do that?

I was looking at Federated queues but not too sure if I can implement mirroring with it.

Thanks. GGarcia

Upvotes: 0

Views: 509

Answers (1)

Horacio Vico
Horacio Vico

Reputation: 21

According to this no, it's not possible:

https://www.rabbitmq.com/blog/2013/10/23/federated-queues-in-3-2-0/

'Federated queues will only ever contain a given message in one location; there's no mirroring.'

I've been thinking, what about implementing federated queues, and providing HA to every node using a 'low level' solution like Pacemaker/DRBD?

https://www.rabbitmq.com/pacemaker.html

Haven't tried it, it's just an idea.

The other thing to try is maybe using sharding instead of federated queues, here it says it should be possible to mirror its sharded queues:

https://github.com/rabbitmq/rabbitmq-sharding/issues/12

Good luck!

Upvotes: 2

Related Questions