CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaarchitectureevent-handlingamazon-sqs
Dima Boyko
Dima Boyko

Reputation: 1

Is there AWS solution to sort out SQS queue?

I have this architectural dilemma, and thought, maybe this is solved problem, or solvable multiple ways. I've got an SQS queue (one of many), which is triggering a Lambda function (one of many). This queue requires slightly different processing of messages, based on one key in the payload. What would be the best way for sorting out the queue messages before actual processing by Lambda?

Should it be separate Lambda, which will check for the key, and then place the message into separate queue, which will trigger corresponding Lambda?

Should it be just bunch of if statements in primary Lambda?

Is there maybe automated way to deal with such situation?

Thanks!

Upvotes: 0

Views: 1368

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269081

It appears that your situation is:

  • An external process sends messages to an Amazon SNS topic
  • An Amazon SQS queue is subscribed to the SNS topic
  • An AWS Lambda function is subscribed to the SQS queue
  • There are various sub-types of messages, each of which should be processed slightly differently

One option is to use Amazon SNS Message Filtering, which can deliver messages differently based upon a message attribute. It could, for example, send a subset of messages to an SNS topic or Lambda function. However, this would require the messages being sent to the SNS topic to have a message attribute defined.

If this is not the case, then possible options are:

  • Use one Lambda function to process all message types (You can include quite complex code in multiple files within a Lambda function!), or
  • Use a Lambda function to determine the message sub-type and then send it to a specific Lambda function (or, send it via an SQS queue to a Lambda function for added resiliency)

Upvotes: 2

Related Questions