CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-lambdaamazon-sqs
Fakeer
Fakeer

Reputation: 195

Triggering lambda from s3 vs Triggering lambda from sqs

I am working on a requirement where you need to submit forms into the RDS database. The forms should not be greater than 20KB. There can be spikes for form submission and we might need to support 300,000 concurrent users .

There are a couple of options i can think of

1) Client -> Api gateway -> lambda -> S3 (trigger) -> lambda ->RDS

2) Client -> Api gateway -> lambda -> SQS (trigger) -> lambda ->RDS

In the 2nd approach, the lambda (in bold) would also be storing in S3 (not shown) for persistent storage of form in case i need to go back and re-process them.

Things we need to consider are

1) Write throughput (lambda writing into s3 or sqs should be fast)

2) Persistent storage (i can go back and replay the messages)

3) In case insert fails in RDS , things like DLQ

Upvotes: 0

Views: 601

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 270154

The only potential bottleneck in this system would be Amazon RDS.

Therefore, using an Amazon SQS queue with would be safer than directly calling Lambda. If the Lambda function triggered from SQS fails, then the message can be retried later or moved to a Dead Letter Queue. It would also be more resilient to spikes of traffic.

Upvotes: 1

Related Questions