CODEWITHSUNDEEP
rabbitmq
Simon Goodman
Simon Goodman

Reputation: 1204

How can I scale RabbitMQ to some consumers

I am using RabbitMQ to send a message to 2 or more consumers. Looking at the simple tutorial I can achieve this very easily.

But I also want to scale some of my consumers and allow some consumers to always process the message and some consumers to only process the message if it has not been processed already by another 'similar' consumer.

    |-------> cA
    |
P --|-------> cB'
    |   
    |-------> cB"

In my example above:

The idea is to process data as fast as possible and, (looking at the picture above), if cB' or cB" need to contact the DB or an API, I want another consumer to pickup the data off the queue as fast as possible. But I also do not want that data to be picked up more than once.

I could always add another publisher and have the messages removed off the queue, but I am not sure if this is really the most efficient way to achieve what I am after.

    |-------> cA
    |
P --|-------> p'--|-------> cB'
                  |
                  |-------> cB"

Is this something that could be achieved in RabbitMQ?

Upvotes: 0

Views: 3799

Answers (2)

Michael de Paly
Michael de Paly

Reputation: 11

If I understood your question correctly a setup with 2 queues will solve your problem.

Both queues have to be bound to the exchange(X) with the same routing key.

If you subscribe the queues in the following pattern:

  • cA as the sole subscriber of queue1
  • cB' and cB" sharing queue2

Routing and Subscription Overview:

         |--<routing key>---queue1-----cA
 P---X---|                          
         |                          |--cB'
         |--<routing key>---queue2--|
                                    |--cB"

The message will:

Upvotes: 1

Milad
Milad

Reputation: 479

Read Consumer Priorities And Also Dead Letter Exchanges

Upvotes: 0

Related Questions