CODEWITHSUNDEEP
rabbitmqrabbitmq-shovel
Tom
Tom

Reputation: 2454

How to configure RabbitMQ bidirectional shovel and avoid infinite loop?

I have 2 RabbitMQ clusters in separate datacenters. At the moment we are shovelling one of the exchanges from an one rabbit to the other but now we need to shovel in the other direction too. How can we achieve this without causing an infinite loop?

A colleague thought we could tick the "Add Forwarding Headers" options and then filter using a Header exchange. However, the headers that get added appear to be nested and I've not found a way to match against the headers?

Upvotes: 4

Views: 2156

Answers (1)

Olivier
Olivier

Reputation: 2691

Seems based on the additional description you provided that:

  • You are using shovel between exchanges
  • Guessing that the exchange type is fanout

I can see two approaches to handle what you're trying to do:

  • Use federation, and configure it in both directions between the exchanges in each cluster
  • Still use shovel, but define a new queue on each exchange and configure a queue to queue shovel

For the shovel option: You have exchange A on cluster 1, exchange A on cluster 2, your main_queue on each cluster

  • Create a secondary_queue on each cluster, bound to exchange A.
  • Create a shovel from secondary queue on cluster 1 to main_queue on cluster 2
  • Create a shovel from secondary queue on cluster 2 to main_queue on cluster 1

Benefit of Federation: simpler, even when you have complex routing configured

Benefit of shovel: better if you have risk of major connection disruption (more than a minute I'd say). In such cases you won't lose messages while it might happen in the Federation case that some messages don't get replicated on both clusters.

Upvotes: 2

Related Questions