RabbitMQ memory management and duplication

Question

I'm currently evaluating the RabbitMQ to manage queue. I was wondering how is RabbitMQ manage queue item in the memory.

in this publisher subscriber example http://www.rabbitmq.com/tutorials/tutorial-three-python.html

is it create queue for each subscriber (consumer) ? for example if I have two consumer then I double the memory usage to store the message?

I'm under the impression that if I attach multiple worker on a queue, then it will became a working queue where each of the consumer received different message.

Let say that I'm building chat server for this. Do I need to create a queue for each consumer ? And every message in the memory would be multiplied by the number of user connected ? or is there only a single message in the memory and every queue have pointer to that message.

Also in the example of topic message. http://www.rabbitmq.com/tutorials/tutorial-five-python.html

let say I have 1kb message . so is there 2kb memory usage for the 2 queue? Q1, Q2 and say the message match all of the binding key.

If I added another queue to listen to let say lazy.blue.* as Q3. Would that create a new queue item in the memory? and duplicate the data ?

Answer 1

From: http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/

If Rabbit needs to save memory and write to disk, only one message is saved.

"Queues themselves decide when and whether to write a message to disk. But a single message can be sent to multiple queues and it is obviously advantageous to make sure each message only gets written to disk once. However, there are two distinct pieces of information here: firstly, the message content itself. This is the same in every queue that the message has been sent to, and should only be written to disk once, regardless of the number of queues it goes to; note that subsequent writes of this do not need to do a value comparison: if the ID of the message is known to the backing store then the message body will match what is already on disk -- message content is never altered by the broker. The second piece of information is the existence of the message in each queue: where in the queue it lies, what its neighbours are, and what its queue-specific status is. This second piece of information is what allows RabbitMQ to start up, recover messages and queues from disk and ensure that the messages in each queue are in the same order as when RabbitMQ was shut down.

Thus RabbitMQ's default backing store consists of a node-global message store which is concerned only with writing message contents to disk; and a per queue queue index which uses a very different format for writing per message per queue data to disk "

Answer 2

In the first example, you have one publisher that sends messages to a particular exchange, and that exchange is binded to two queues.

In regards to creating a queue for each consumer(I'm assuming that’s what you mean when you say “subscriber”), that’s entirely up to you. In the first diagram there are no consumers shown, but you can configure a consumer(s) to listen for messages on either one of these queues, or both. The queues (not consumers) will affect your memory; exchanges in comparison are cheap in terms of memory cost.

For the second diagram, yes, creating a third queue “Q3” would create a new queue in memory. I'm a bit confused by your last question, you seem to be asking what would happen if an exchange sends a message that matches all binding patterns? If that happened here, the message would simply be sent to all three queues. Topic-type exchanges (as the exchange in the example is set to) simply route messages to all queues whose binding key matches the message's routing key.

Answer 3

I'm not a rabbitmq hacker but the common sense says that each queue should have a pointer to one unique message instance.