CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaamazon-sqs
rock3t
rock3t

Reputation: 2233

How to pool AWS SQS with AWS Lambda

At the moment I'm are pooling AWS SQS from our back-end and doing business logic once payload is received.

I would like to move this to AWS Lambda and start automating business logic via SQS/SNS.

As I can not subscribe to AWS SQS events, what is the best practice in implementing SQS pooling with Lambda (node.js)?

Upvotes: 1

Views: 897

Answers (3)

mtk
mtk

Reputation: 13709

As I can not subscribe to AWS SQS events,

Why?

Lambda can be triggered on SQS messages. Lambda internally handles the scaling, batching and retries for you. Check this AWS documentation on how to use Lambda with SQS.

Upvotes: 0

H6_
H6_

Reputation: 32788

It is possible to even simplify the whole polling process by using the built-in SQS event source for lambda.

Lambda will automatically scale out horizontally consume the messages in my queue. Lambda will try to consume the queue as quickly and effeciently as possible by maximizing concurrency within the bounds of each service. As the queue traffic fluctuates the Lambda service will scale the polling operations up and down based on the number of inflight messages.

see AWS Blog

Upvotes: 1

fabien
fabien

Reputation: 201

SQS doesn't really work well with Lambda since you cannot automatically trigger Lambda functions from SQS queues messages.

I would rather remove the SQS/SNS logic and go for a DynamoDB Streams based solution that would cover the queueing, archiving & Lambda triggering tasks natively: your producer puts messages in a DynamoDB table while your Lambda is triggered for any new entry with Streams (it's an AWS native mechanism)

Of course a Kinesis based solution may be considered as well.

Upvotes: 1

Related Questions