Message is not routing to dead letter queue when consumer is down

Question

I've a service A which is publishing message to Queue(Q-A). I've a dead letter queue(DLQ) bounded to DLX with DLRK. Queue A is bounded to an exchange(E-A) with a routing key(RA). I've also set x-letter-exchange(DLX) and x-dead-letter-routing-key(DLRK) on Q-A with ttl-per-message on this queue to 60 seconds The DLQ is also set with x-letter-exchange(E-A) and x-dead-letter-routing-key(DLRK) with ttl-per-message on this queue to 60 seconds.

With above configuration I'm trying to route the message to DLQ from Q-A after ttl expires and vice versa. On the consumer side which is another service, I throw AMQPRejectAndDontRequeueException with defaultRequeueRejected set to fals.

The above configuration works fine when the consumer is up and throws the
exception. But I'm trying to limit my queue size to 1 and then publish 3 messages to the Q-A and also shutting down the consumer. I see all the three messages placed in both Q-A and DLQ and eventually all the messages are dropped. But if I don't set the queue limit to 1 or start the consumer, everything works fine.

I've also set the x-overflow to reject-publish and when there is overflow, I get a nack at the publisher and then I've a scheduler which publish it again to Q-A.

Note: Both exchanges are Direct and I'm using routing keys to bind it to respective queue.

Kindly, let me know if I'm missing something here and let me know need to share my config

Answer 1

I may have arrived at this too late, But I think I can help you with this.

Story: You want a retry queue to send dead messages to and retrieve and re-queue them in the main queue after a certain amount of time.

Solution:

1. Declare your main queue and bind it to an exchange. We call them main_queue and main_exchange and add this feature to the main_queue: x-dead-letter-exchange: retry_exchange
2. Create your retry queue and bind it to another exchange. We call these retry_queue and retry_exchange and add these features to the retry queue: x-dead-letter-exchange: main_exchange and x-message-ttl: 10000

With this combination, dead messages from main_queue will be sent to retry_queue and after 10 seconds they will be sent again to the main_queue which will they last indefinitely until a consumer declares them dead.

Note: This method works only if you publish your messages to the exchange and not directly in the queue.

Answer 2

After digging through, I think i finally found the answer from the link Dead-lettering dead-lettered messages in RabbitMQ answer by pinepain

It is possible to form a cycle of dead-letter queues. For instance, this can happen when a queue dead-letters messages to the default exchange without specifiying a dead-letter routing key. Messages in such cycles (i.e. messages that reach the same queue twice) will be dropped if the entire cycle is due to message expiry.

So I think to solve the problem I need to create another consumer to consume from dead letter queue and publish it back to original queue from the consumer and not directly ttl from the dead letter queue. Please correct me if my understanding is right.