CODEWITHSUNDEEP
amazon-ec2rabbitmq
Santhosh S
Santhosh S

Reputation: 1159

RabbitMQ HA Cluster on Amazon EC2

In Amazon VPC, on two nodes I have installed rabbitmq

On Node 1, I ran the following commands

#Node 1
/etc/init.d/rabbitmq-server stop
rabbitmq-server -detached
rabbitmqctl start_app
rabbitmqctl set_policy HA '^(?!amq\.).*' '{"ha-mode": "all"}'

On Node 2, I ran the following commands to setup the cluster

/etc/init.d/rabbitmq-server stop
rabbitmq-server -detached
rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@<PrivateIP>
rabbitmqctl start_app
rabbitmqctl set_policy HA '^(?!amq\.).*' '{"ha-mode": "all"}'

RabbitMQ nodes are behind a Elastic Load Balancer. I ran a java program to keep pushing messages into rabbitmq.

Case 1: rabbitmqctl list-queues -- showed the quename and queue message count same while the java program was pushing messages to the queue.

Case 2: I stopped rabbitmq on node 2 and then started it again. Checked the cluster status and queue message counts. The message count was correct ( 3330 on both node 1 and node 2 )

Case 3 : I stopped rabbitmq on node 1 while the java program was pushing messages to the queue. I checked the queue message count in node 2 , count was 70. I started rabbitmq on node 1, and then checked queue count was 75.

I want to setup a rabbitmq high availability cluster and ensure no message loss. I have enabled sync_queue on rabitmq start in /etc/init.d/rabbitmq-server.

Appreciate if you can point out, why the message counts dropped from approx 3330 to 70. And also what's the best way to setup and ensure HA.

Upvotes: 0

Views: 1242

Answers (2)

Gabriele Santomaggio
Gabriele Santomaggio

Reputation: 22682

If you don't want lose messages you should considerer to use tx-transtaction

channel.txSelect();
channel.basicPublish("", youQueue, MessageProperties.PERSISTENT_TEXT_PLAIN,
message.getBytes());
channel.txCommit();

This could be kill the performance, if you have a high messages rate.

Visit http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/

Upvotes: 1

ldx
ldx

Reputation: 4074

A few tips:

  • Does your app use publisher confirms? If you don't want to lose messages, it should.
  • Is automatic syncing of queues enabled? If not, you have to manually initiate queue syncing for any queue.
  • You should not restart any node while queues are being synced, or messages might be lost.

Upvotes: 2

Related Questions