CODEWITHSUNDEEP
loggingrabbitmq
Chris McKenzie
Chris McKenzie

Reputation: 3851

How to log all Rabbit MQ messages?

We're using a RabbitMQ server for messaging between applications. We have a need to create a central log for all amqp messages coming into the Rabbit server. Our purpose is not temporary debugging, but rather auditability. Ideally, I'd be able to log to a specified file to begin with, and later log to an external system such as Logly or Splunk.

I've explored turning on Firehose and using the tracing plugin, but the queues in question are not durable. I'm also not sure if these solutions will work if new queues and exchanges are added to the virtual host after logging hast started. These tools seem designed more for temporary debugging that what I need.

I'd love to hear your ideas. At this point I'm worried I'll have to setup a network monitor to intercept and log the messages before they reach Rabbit.

Upvotes: 22

Views: 44914

Answers (2)

Josh Withee
Josh Withee

Reputation: 11386

The RabbitMQ Tracing Plugin extends the functionality of the built-in trace feature, allowing you to send traced messages straight to a log file (which can be json format, if desired).

Regarding performance impact: In the original rabbitmq-tracing repo, the following was stated in 2015 or earlier: "On a few year old developer-grade machine, rabbitmq-tracing can write about 2000 msg/s to a log file."

So performance impact should not be an issue except in very high throughput situations.

Upvotes: 1

old_sound
old_sound

Reputation: 2313

Why would you say that the firehose won't work? You can create the queues for listening as you want, i.e.: durable and so on. Then if you need every message, you can do a binding using the # character.

See: http://www.rabbitmq.com/firehose.html

Performance will drop as explained, but I wouldn't ditch the solution without testing performance.

Upvotes: 6

Related Questions