CODEWITHSUNDEEP
javaamazon-web-servicesamazon-sqs
Madhukar R
Madhukar R

Reputation: 31

Amazon SQS - FIFO Queue message request, inconsistent receives

I have a FIFO SQS queue, with visibility time of 30 seconds. The requirement is to read messages as Quickly as possible and clear the queue.

I have code in JAVA in a fashion shown below ( this is just a representation of idea only, not complete code ):

//keep getting messages from FIFO and process them ASAP
while(true)
{
     List<Message> messages = 
     sqsclient.receiveMessage(receiveMessageRequest).getMessages();
    //my logic/code here to process these messages and delete them ASAP
}

In the while loop as soon as the messages are received, they are processed and removed from the queue. But, many times the receiveMessageRequest does not give me messages (returns zero messages).

Also, the messages limitation is only 10 at a time during receive from SQS, which is already an issue, but due to these zero receives, the queues are piling up.

I have no clue why this is happening. The documentation exactly is not clear on this part (or Am I missing in terms of the configuration of the queue?)

Please help!

Note: 1. My FIFO Queue always has messages in this scenario, so there is no case of Queue having zero messages and receive request returning zero

2. The processing and delete times are also Less than the visibility timeout.

Thanks.

Update: I have started running multiple consumers for processing the FIFO queue. Clearly, one consumer is not coping up with the inflow of messages. I shall update in few days how multiple consumers are performing. Thanks

Upvotes: 3

Views: 1801

Answers (1)

user2129038
user2129038

Reputation: 184

You have to first make sure that all messages you received are deleted within VisibilityTimeout. If you are using DeleteMessageBatch for deletion make sure that all 10 messages are deleted.

Also, how did you queue messages when you enqueue them? Order of messages are guaranteed only in a single message group. This also means that if you set the same group id to all messages, you are limited to a single consumer so that order of messages are preserved for sure. Even if use multiple consumers, all messages that belong to a same group becomes invisible to other consumers until visibility timeout expires.

Upvotes: 3

Related Questions