CODEWITHSUNDEEP
amazon-web-servicesamazon-sqs
Christopher Stevens
Christopher Stevens

Reputation: 1248

Is it possible to estimate where in line an SQS message is in a queue?

My hunch is that the answer is "no", which is an answer I'll accept. May I get your confirmation?

I have an electronics project that thanks donors one at a time, reading from an SQS queue that gets populated with successful donations. At the moment I have one "thanking device", but this could scale if demand grows, making Amazon SQS a good candidate for managing queue assignments.

With the device only being able to thank one person at a time, the SQS queue could potentially fill up quickly at times. It would be nice to give a donor a status update that their donation is number 5 in the queue to be thanked for example. Is this "number in line" status update option possible with SQS?

When adding an item to a queue, it returns an assigned sequence number as part of the response:

{
  ResponseMetadata: { RequestId: 'aaaaaaaa-9ee6-58e4-a511-7f731ebe8145' },
  MD5OfMessageBody: 'aaaaaaaa80be1add497c0685c1e426a5',
  MessageId: 'aaaaaaaa-f4e2-4ff5-a3ec-9d017585f60b',
  SequenceNumber: '18835430100687855616' 
}

This number isn't sequential in a single queue, possibly ruling out using this number as a helper for spot in line. As an example here are two sequence numbers returned after adding two items to the queue within a couple of seconds of each other:

18835430099287791616
18835430100687855616

I could use get-queue-attributes to get an estimated total of donations in the queue (ApproximateNumberOfMessages) at the time a donation succeeded. That would help get an initial estimate.

Brainstorming beyond acceptable "no" answer:

Perhaps I could store a counter somewhere that would update the approximate number of other donors needing thanked first, per donor session? At the moment I see a processed count being stored in single-record dynamo-db table (maybe not best use case) or stored in S3 as a JSON doc (although this could get messy at scale, maybe not). When a donation succeeds, the donor receives the initial count of processed thanks and estimated queue length where delta calculations can happen at intervals.

I'm open to thoughts.

Upvotes: 0

Views: 140

Answers (1)

jarmod
jarmod

Reputation: 78553

I'm not aware of any supported mechanism to determine where in an SQS queue a specific message is at any given time.

But, as you say, you can retrieve an approximation of the number of visible messages in the queue at the time the message was added to the queue, using GetQueueAttributes. You could store this information in DynamoDB and perhaps also maintain an indicator of your approximate processing rate of SQS messages (or leverage SQS queue metrics from CloudWatch). That could potentially be used to give an estimate of how long it will take to process the message.

Upvotes: 1

Related Questions