CODEWITHSUNDEEP
amazon-web-servicesamazon-sqs
Loc12342
Loc12342

Reputation: 43

AWS SQS Selective Polling Pattern

I have a system where I publish updates to a shared topic meant for specific consumers. I noticed messages getting stuck in the queue due to a lack of selective listening in SQS consumers, so messages are being hijacked.

Example:

Given: Message{destination: A, payload: 1234}

Given: ConsumerA, & ConsumerB

I expect Message to be processed by ConsumerA. However, it gets hijacked by Consumer B continuously. It receives the message, then refuses to process it since the destination field doesn't match, leading to the visibility timeout to expire, and the message put back on the queue.. but due to the nature of SQS, ConsumerB has an equal chance of picking the message again.

My question is, what patterns are used to solve this type of issue? I'm considering creating a queue per consumer but it has drawbacks specific to the system im working on.

If I could only listen for messages with matching attributes, problem solved, but that's seemingly not the case.

Is there any other way?

Upvotes: 2

Views: 317

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269081

Sharing a single Amazon SQS queue is not an appropriate architecture for your use-case.

If you want your consumers to be able to 'request' a message from a particular subset, you should either use separate SQS queues or use a database. You could even store objects in Amazon S3 as a form of noSQL database.

Having consumers grab messages and then 'send them back' to the queue is not compatible with the design of the Amazon SQS service.

Upvotes: 2

Related Questions