CODEWITHSUNDEEP
amazon-dynamodbamazon-redshiftdatabase-replicationtungsten-replicator
chaithu
chaithu

Reputation: 519

Is there a way to sync table data from DynamoDB to Redshift?

We are currently using Amazon RDS for our database and we are planning to move to DynamoDB to scale our messaging DB. But a lot of analytics has been setup on MySQL tables which we want to keep using.

What is the best way to sync data from DynamoDB to Redshift ?

Upvotes: 1

Views: 2931

Answers (1)

Henryk Konsek
Henryk Konsek

Reputation: 9168

For this kind of problems (synchronization of source database into Redshift) you have basically two common solutions:

  • (A) create application that periodically polls input database, detects delta and ingest detected delta data into Redshift
  • (B) use AWS Kinesis Firehose

Option B is much easier to implement, as you basically use dedicated AWS service for this purpose. However it is suitable only for source databases supported by Firehose. Luckily for you, DynamoDB is supported as input source for Firehose. Also you need to calculate an extra cost associated with Kinesis itself, Lambda used for record conversions, etc. to be sure that this option is valid for you from economical point of view. See this article to see how you can connect DynamoDB streams with Kinesis using Firehose.

Option A is usually a valid solution for source databases not supported by Firehose.

Upvotes: 1

Related Questions