CODEWITHSUNDEEP
node.jsrabbitmqamqp
Srle
Srle

Reputation: 10496

Using publisher confirms with RabbitMQ, in which cases publisher will be notified about success/failure?

Quoting the book, RabbitMQ in Depth:

A Basic.Ack request is sent to a publisher when a message that it has published has been directly consumed by consumer applications on all queues it was routed to or that the message was enqueued and persisted if requested.

Confused with Has been directly consumed, does it mean when consumer send ack to broker publisher will be informed that consumer process message successfully? or it means that publisher will be notified when consumer just receive message from the queue?

or that the message was enqueued and persisted if requested. Is this like conjuction or publisher will be informed when either of those happens? (In that case publisher would be notified twice)

Using node.js and amqplib wanted to check what is happening actually:

// consumer.js
amqp.connect(...)
.then(connection => connection.createChannel())
.then(() => { assert exchange here })
.then(() => { assert queue here })
.then(() => { bind queue and exchange here })
.then(() => {
  channel.consume(QUEUE, (message) => {
    console.log('Raw RabbitMQ message received', message)

    // Simulate some job to do
    setTimeout(() => {
      channel.ack(message, false)
    }, 5000})

  }, { noAck: false })
})

// publisher.js
amqp.connect(...)
.then(connection => connection.createConfirmChannel())
.then(() => { assert exchange here })
.then(() => {
  channel.publish(exchange, routingKey, new Buffer(...),{}, (err, ok) => {
    if (err) {
      console.log('Error from handling confirmation on publisher side', err)
    } else {
      console.log('From handling confirmation on publisher side', ok)
    }
  })
})

Running the example, i can see following logs:

From handling confirmation on publisher side undefined
Raw RabbitMQ message received
Time to ack the message

As far as i see, at least by this log, publisher will be notified only when message was enqueued? (So having consumer acking the message will not influence publisher in any way)

Quoting further:

If a message cannot be routed, the broker will send a Basic.Nack RPC request indicating the failure. It is then up to the publisher to decide what to do with the message.

Changing the above example, where i only changed the routing key of the message to something that should not be routed anywhere (there are no bindings that would match routing key), from logs i can see only following.

From handling confirmation on publisher side undefined

Now i'm more confused, about what publisher is notified exactly here? I would understand that it receive an error, like Can't route anywhere, that would be aligned with quote above. But as you can see err is not defined and as side question even if amqplib in their official docs are using (err, ok), in no single case i see those defined. So here output is same like in above example, how one can differ between above example and un-routable message.

So what im up to here, when exactly publisher will be notified about what is happening with the message? Any concrete example in which one would use PublisherConfirms? From logging above, i would conclude that is nice to have it in cases where you want to be 100% sure that message was enqueued.

Upvotes: 9

Views: 11823

Answers (3)

Wes Tyler
Wes Tyler

Reputation: 23

I'm not entirely sure about the notification on ack/nack question, but check out the BunnyBus Node library for a simpler api and RabbitMQ management :)

https://github.com/xogroup/bunnybus

const BunnyBus = require('bunnybus');
const bunnyBus = new BunnyBus({
    user: 'your-user',
    vhost: 'your-vhost', // cloudamqp defaults vhost to the username
    password: 'your-password',
    server: 'your.server.com'
});

const handler = {
    'test.event': (message, ack) => {

        // Do your work here.

        // acknowledge the message off of the bus.
        return ack();
    }
};

// Create exchange and queue if they do not already exist and then auto connect.
return bunnyBus.subscribe('test', handler)
    .then(() => {

        return bunnyBus.publish({event: 'test.event', body: 'here\'s the thing.'});
    })
    .catch(console.log);

Upvotes: 0

Srle
Srle

Reputation: 10496

After searching again and again i have found this http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/

The basic rules are as follows:

  1. An un-routable mandatory or immediate message is confirmed right after the basic.return
  2. transient message is confirmed the moment it is enqueued
  3. Persistent message is confirmed when it is persisted to disk or when it is consumed on every queue.

If more than one of these conditions are met, only the first causes a confirm to be sent. Every published message will be confirmed sooner or later and no message will be confirmed more than once.

Upvotes: 8

Gabriele Santomaggio
Gabriele Santomaggio

Reputation: 22682

by default publishers don't know anything about consumers.

PublisherConfirms is used to check if the message reached the broker, but not if the message has been enqueued.

you can use mandatory flag to be sure the message has been routed see this https://www.rabbitmq.com/reliability.html

To ensure messages are routed to a single known queue, the producer can just declare a destination queue and publish directly to it. If messages may be routed in more complex ways but the producer still needs to know if they reached at least one queue, it can set the mandatory flag on a basic.publish, ensuring that a basic.return (containing a reply code and some textual explanation) will be sent back to the client if no queues were appropriately bound.

Upvotes: 7

Related Questions