CODEWITHSUNDEEP
aws-lambdaamazon-dynamodb-streams
DJ Spiess
DJ Spiess

Reputation: 993

DynamoDB Stream to Lambda slow/unsuable

I've connected a lambda to a DyDB table via a stream. When a record is written to the table, it triggers the lambda. The traffic is very bursty, so nothing might happen for a while, then I'll write several thousand records.

What I'm seeing is a few lambda instances will be triggered, but not enough to handle the burst. Then at random times, the number of lambda instances will jump an order of magnitude or two (from 2 to 90 or more), and it will catch up. The problem is the jump might not occur for 30 minutes or more.

I'm seeing the records written to the table very quickly (seconds). The processing of 20 records by the lambda shouldn't take more than 2 minutes. It seems like the lambdas are spending most of their time sitting around waiting for records to show up. The record key for the table is a GUID.

Things I've tried

Is there a better pattern to be using? Should I skip the stream and just write SNS messages? I don't care about order, but would prefer to not run the job more than once.

Upvotes: 3

Views: 2478

Answers (1)

DJ Spiess
DJ Spiess

Reputation: 993

So here's what I found out.

It looks like the problem is contention on the DynamoDB stream by the lambda instances.

My solution was to skip the DynamoDB stream and not use it, and post to an SNS queue. The lambdas pick up the messages, and scale much better. Times have gone from hours to seconds.

Upvotes: 2

Related Questions