AWS DynamoDB to S3 Hourly Backup

Question

I have gone through couple of stackoverflow questions regarding hourly backups from DDB to S3 where the best solution turned out to be to enable DDB Stream, subscribe lambda function and push to S3.

I am trying to understand if directly pushing from Lambda to S3 is fine or from Lambda to Kinesis Firehose and then to S3. Can someone share what is the advantage if we introduce Firehose in between. We anyways trigger lambda only after specific batch window that implies we are already buffering there.

Thanks in advance.

Answer 1

You can write a Lambda function that reads a DynamoDB table, gets a result set, encodes the data to some format (ie, JSON), then place that JSON into an Amazon S3 bucket. You can use scheduled events to fire off the Lambda function on a regular schedule.

Here in AWS tutorial that shows you how to use scheduled events to invoke a Lambda function:

Creating scheduled events to invoke Lambda functions

This AWS tutorial also shows you how to read data from an Amazon DynamoDB table from a Lambda function.

Answer 2

Firehose gives you the possibility to convert and compress your data. In addition you can directly attach a Glue Metadata table, so you can query your data with Athena.