CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-lambdaamazon-dynamodbamazon-dynamodb-streams
user3293898
user3293898

Reputation: 271

DynamoDB Streams to S3

I am using Data Pipeline (DP) for daily backups of DynamoDB, however, I would like to do incremental backups of the data that is missed by DP runs (updates between DP runs). To accomplish that, I would like to use DynamoDB Streams + Lambda + S3 to bring real-time DynamoDB updates to S3. I understand how DynamoDB streams work, however, I am struggling with creating a Lambda function that writes to S3 and say rolls a file every hour.

Has anyone tried it?

Upvotes: 11

Views: 13568

Answers (1)

AnubhavJain
AnubhavJain

Reputation: 129

Its an hour job dude,What you need to do is

  1. Enable Dynamo DB update Stream and attach aws provided lambda function https://github.com/awslabs/lambda-streams-to-firehose
  2. Enable Firehose stream and use above function to stream outs records in firehose.
  3. Configure Firehose to dump the records to S3.

done.

Upvotes: 12

Related Questions