CODEWITHSUNDEEP
amazon-web-servicesamazon-sqs
sdgfsdh
sdgfsdh

Reputation: 37025

When should I delete messages in SQS?

My application consists of:

The workers have the following logic: 

 1. Wait for message from SQS queue
 2. Perform task described in message
 3. Delete the message from the SQS queue
 4. Go to (1)

I want each message to be received by only one worker to avoid redundant work.

Is there a mechanism to mark a message as "in progress" using SQS, so that other pollers do not receive it?

Alternatively, is it appropriate to delete the message as soon as it is received? 

 1. Wait for message from SQS queue
 2. Delete the message from the SQS queue
 3. Perform task described in message
 4. Go to (1)

If I follow this approach, is there a way to recover received but unprocessed messages in case a worker crashes (step (3) fails)?

This question is specific to Spring, which contains all sorts of magic.

Upvotes: 5

Views: 3405

Answers (2)

Rodrigo Murillo
Rodrigo Murillo

Reputation: 13632

An SQS message is considered to be "inflight" after it is received from a queue by a consumer, but not yet deleted from the queue. These messages are not visible to other consumers.

In SQS messaging, a message is considered in "inflight" if:

  1. You the consumer have received it, and
  2. the visibility timeout has not expired and
  3. you have not deleted it.

SQS is designed so that you can call ReceiveMessage and a message is given to you for processing. You have some amount of time (the visibility timeout) to perform the processing on this message. During this "visibility" timeout, if you call ReceiveMessage again, no worker will be returned the message you are currently working with. It is hidden.

Once the visibility timeout expires the message will be able to be returned to future ReceiveMessage calls. This could happen if the consumer fails in some way. If the process is successful, then you can delete the message.

The number of messages that are hidden from ReceiveMessage call is the "inflight" number. Currently a SQS queue is set by default to allow a max of 120,000 messages to be "inflight".

http://docs.amazonwebservices.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/AboutVT.html

Upvotes: 9

sreerag
sreerag

Reputation: 161

There will be ReciepientHandle String which will be sent with the message. It will be having a expiry time based on queue visibility timeout.

You can use the this ReciepientHandle to delete message from queue

Upvotes: 0

Related Questions