CODEWITHSUNDEEP
linuxrabbitmqbackupreliability
Steven
Steven

Reputation: 177

How to make RabbitMQ more reliable? Is it possible to have it backup its data (messages) periodically?

I want to ensure no will be lost in RabbitMQ if case it crashes. Is is possible to somehow have RabbitMQ dump or backup all the queues it has on disk?

I also have heard that once it receives a messages and even if it crashes, when it's up the message will be available.

So what do you recommend to make the process more reliable?

Upvotes: 2

Views: 950

Answers (2)

Gabriele Santomaggio
Gabriele Santomaggio

Reputation: 22760

The best way is to create an RabbitMQ cluster and then configure the HA policies.

When you configure the HA, the messages are replicated to the other nodes. If one node crashes you can accessthe other node(s).

You can also backup the Mnesia directory that usually contains configuration and messages, but some messages can stay to the cache. To be sure you should stop the rabbitmq application.

I'd try using the HA.

Another way to backup continuously the messages is using Shovel plugin

As mentioned by @Milind using a persistent message is an best practise.

edit

You cloud use the Tx Transaction please read here: https://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/

Tx can solve your problem, but can have a big impact to the performance.

Upvotes: 0

Milind
Milind

Reputation: 99

In RabbitMQ, this can be achieved using the durability of the queues and messages. Queue durability means that its configuration will be stored on the disk. And when RabbitMQ recovers after a crash, the queue will be restored. It can be done thus :

channel.queue_declare(queue='myQueue', durable=True)

What you seek is more closely related to message durability. Publish all your messages with the delivery_mode flag set to 2. The default value is 1, which makes messages reside only in the RAM. See this for code samples in Python. Excerpt below.

channel.basic_publish(exchange='',
                  routing_key='task_queue',
                  body=message,
                  properties=pika.BasicProperties(
                     delivery_mode = 2, # make message persistent
                  ))

Please note that enabling message durability has its effects on performance. When set to 2, RabbitMQ broker has to do the additional work of writing messages to and reading them from the disk.

Upvotes: 2

Related Questions