CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaamazon-dynamodb
Joey Yi Zhao
Joey Yi Zhao

Reputation: 42500

How should I configure `Concurrent batches per shard` when using lambda to read stream from Dynamodb?

I have a Dynamodb table enabled stream to a lambda function. I have configured Concurrent batches per shard to be 1 which I expect the maximum concurrency of my lambda is only 1. But when monitoring the lambda metrics, I see there are multiple lambda instance running at the same time.

My question is that how Concurrent batches per shard relate to a lambda concurrency?

Does one shard include all records relate to the same partition key?

My other configuration looks like:

enter image description here

If one shard includes update records from the same partition key, then if I configure more than 1 concurrencies per shard, does it mean multiple lambda invocations includes records from the same partition key? If yes, how does it handle processing records in order?

For example, I have 10 updates on the same record in a Dynamodb table at the same time. If there are more than 1 concurrencies per shard, will the 10 updates come to different lambda invocation via stream?

Does the shard also have anything to do with sort key?

Upvotes: 0

Views: 842

Answers (1)

Leeroy Hannigan
Leeroy Hannigan

Reputation: 19793

You have multiple invocations as you have multiple shards in the stream. Stream shards are tightly bound to DynamoDB partitions and therefore also bound to Lambda invocations.

DynamoDB Partitions : Stream shards : Lambda invocations

1 : 1 : 1

Therefore even with a concurrency of 1 per shard, you will still have n invocations or the lambda, where n = no of partitions/shards.

Upvotes: 1

Related Questions