CODEWITHSUNDEEP
phprabbitmqamqpphp-amqplib
Aistis
Aistis

Reputation: 4053

How to send unacked messages to DLX after TTL?

I have rabbit queue my_queue which is configured with DLX sending messages to my_queue_dlx. A test with message-ttl works as expected. A message which stays in a queue for ttl and later moved to DLX if it is not consumed.

I have a PHP consumer with php-amqplib which consumes a message and after a successful handling sends ack.

Problem: It may happen that this PHP consumer will be killed by OS or PHP runtime itself because of high memory usage and this is unknown in advance before handling a message. I basically want to have a unacknowledged message handled by PHP consumer and if it got killed because of OOM, the message would expire and go to DLX.

Is there a way to set unacknowledged message timeout/expiration, so it would go to DLX?

Current my_queue example:

my_queue:
    x-queue-mode: lazy
    x-max-length-bytes: 53687091200 # limited to 50GBs
    x-overflow: drop-head
    x-dead-letter-exchange: some_exchange
    x-dead-letter-routing-key: my_queue_dlx

Upvotes: 0

Views: 497

Answers (1)

Sammitch
Sammitch

Reputation: 32232

I think you want consumer_timeout rather than TTL.

Though this will likely cause messages to be re-delivered until they either hit a configured TTL or are otherwise rejected by a consumer. This would fit your case of "consumer gone away", but not necessarily a "poison message" scenario. I would suggest using both TTL for an overall deadline for message processing and consumer_timeout for the concern of an individual consumer.

If you want to fix a message to a certain number of deliveries, then I believe that that setting is only available via Quorum Queues' delivery-limit setting. However Quorum Queues have their own set of unique caveats, as opposed to "classic" queues.

TLDR:

  • TTL: "Messages must be handled within this time frame, one way or another."
  • consumer_timeout: "The consumer must handle messages within this amount of time."
  • delivery-limit: "Only attempt X deliveries of any given message in this queue, with some additional caveats."

Upvotes: 0

Related Questions