CODEWITHSUNDEEP
aws-lambdaamazon-dynamodbaws-step-functionsamazon-dynamodb-streams
nfornicole
nfornicole

Reputation: 47

How to avoid concurrent requests to a lambda

I have a ReportGeneration lambda that takes request from client and adds following entries to a DDB table.

Customer ID <hash key>
ReportGenerationRequestID(UUID) <sort key>
ExecutionStartTime
ReportExecutionStatus < workflow status>

I have enabled DDB stream trigger on this table and a create entry in this table triggers the report generation workflow. This is a multi-step workflow that takes a while to complete. Where ReportExecutionStatus is the status of the report processing workflow. I am supposed to maintain the history of all report generation requests that a customer has initiated.

Now What I am trying to do is avoid concurrent processing requests by the same customer, so if a report for a customer is already getting generated don’t create another record in DDB ?

Option Considered :

Issue: If customer clicks in a split second to generate report, two lambdas can be triggered, causing 2 entires in DDB and two parallel workflows can be initiated something that I don’t want.

Can someone recommend what will be the best approach to ensure that there are no concurrent executions (2 worklflows) for the same Report from same customer. In short when one execution is in progress another one should not start.

Upvotes: 1

Views: 1096

Answers (2)

peter
peter

Reputation: 15149

You can use ConditionExpression to only create the entry if it doesn't already exist - if you need to check different items, than you can use DynamoDB Transactions to check if another item already exists and if not, create your item.

Those would be the ways to do it with DynamoDB, getting a higher consistency.

Another option would be to use SQS FIFO queues. You can group them by the customer ID, then you wouldn't have concurrent processing of messages for the same customer. Additionally with this SQS solution you get all the advantages of using SQS - like automated retry mechanisms or a dead letter queue.

Upvotes: 1

Jens
Jens

Reputation: 21540

Limiting the number of concurrent Lambda executions is not possible as far as I know. That is the whole point of AWS Lambda, to easily scale and run multiple Lambdas concurrently.

That said, there is probably a better solution for your problem using a DynamoDB feature called "Strongly Consistent Reads"

By default reads to DynamoDB (if you use the AWS SDK) are eventually consistent, causing the behaviour you observed: Two writes to the same table are made but your Lambda only was able to notice one of those writes.

If you use Strongly consistent reads, the documentation states:

When you request a strongly consistent read, DynamoDB returns a response with the most up-to-date data, reflecting the updates from all prior write operations that were successful.

So your Lambda needs to do a strongly consistent read to your table to check if the customer already has a job running. If there is already a job running the Lambda does not create a new job.

Upvotes: 0

Related Questions