CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaamazon-dynamodbamazon-dynamodb-streams
zkello
zkello

Reputation: 239

Scheduled reading of DynamoDB Stream

I am looking for a way to batch-read updates from DynamoDB in scheduled intervals.

For instance, every 10 minutes I want to be able to read all the updates to the DynamoDB table that occurred since the previous read.

I understand the DynamoDB Streams can be setup to trigger a Lambda Function. Is there anyway for Lambda to batch all the updates over a certain time interval? To be processed all at once?

Upvotes: 3

Views: 769

Answers (1)

johni
johni

Reputation: 5568

Yes.

You need to set a cloud watch alarm (every 10 minutes, like a cronjob, and configure it as a trigger to your lambda.

However (!), you will need to write the code that reads the dynamodb stream and that is going to be an it of a challenge.

You will need to persist somewhere (another dynamodb table, S3 or redis) the last position processed in the dynamodb stream - so you won't be handling the same update twice.

I highly recommend you use the default topology, and set the trigger to be dynamodb, then your lambda will get as input the updated records. AWS manages for you the position on the stream and that is (unlike to other option) is a scalable solution.

Upvotes: 3

Related Questions