CODEWITHSUNDEEP
amazon-web-servicesamazon-sqs
alexanoid
alexanoid

Reputation: 25770

AWS SQS message retention period

According to documentation the maximum SQS message retention period(MessageRetentionPeriod) is 14 days. After that time message will be deleted from the queue.

Is any way with SQS to not lose these messages after their retention period expired? For example, it is not clear or is it possible to use Dead Letter Queue for this purpose?

Upvotes: 28

Views: 46234

Answers (2)

bibilouis
bibilouis

Reputation: 63

Contrary to some answers or blog posts you can find on the web:

The RedrivePolicy is not impacting what will happen to your message if they expire (reached the RetentionPeriod of the queue).

AWS documentation is stating the message is deleted and there is never a mention about RedrivePolicy playing a role.

https://aws.amazon.com/sqs/faqs/#Limits_and_restrictions

Q: How long can I keep my messages in Amazon SQS message queues?

Longer message retention provides greater flexibility to allow for longer intervals between message production and consumption.

You can configure the Amazon SQS message retention period to a value from 1 minute to 14 days. The default is 4 days. Once the message retention quota is reached, your messages are automatically deleted.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SetQueueAttributes.html

MessageRetentionPeriod – The length of time, in seconds, for which Amazon SQS retains a message. Valid values: An integer representing seconds, from 60 (1 minute) to 1,209,600 (14 days). Default: 345,600 (4 days). When you change a queue's attributes, the change can take up to 60 seconds for most of the attributes to propagate throughout the Amazon SQS system. Changes made to the MessageRetentionPeriod attribute can take up to 15 minutes and will impact existing messages in the queue potentially causing them to be expired and deleted if the MessageRetentionPeriod is reduced below the age of existing messages.

So DLQ can't help you here, and after retention period is passed, your message is simply gone, I am afraid.

PS: One can argue the absence of indication is not enough to prove it, and we'll be right.

I am basing this answer also on a real life production events that occurred to me last week.

Upvotes: 2

Piyush Patil
Piyush Patil

Reputation: 14523

Well 14 days is the max limit you can keep the message. After 14 days you can move that message to S3 Bucket for backup. Also there is a hack you can do with DLQ.

Here is a quick hack where you send that message back to the main queue. This is definitely not the best or recommended option.

  1. Set the main SQS queue as the DLQ for the actual DLQ with Maximum Receives as 1.
  2. View the content in DLQ (This will move the messages to the main queue as this is the DLQ for the actual DLQ)
  3. Remove the setting so that the main queue is no more the DLQ of the actual DLQ

Upvotes: 16

Related Questions