CODEWITHSUNDEEP
amazon-web-servicesamazon-s3synchronizationamazon-rds
Ignasi Mañé
Ignasi Mañé

Reputation: 53

Sync Amazon RDS (PostgreSQL) to S3 in near real time

I'm wondering whether it is possible to easily sync an Amazon RDS PostgreSQL database to Amazon S3 in near real time so that data can be used with Amazon Athena, just as read replicas do.

We have several RDS database and we would like to consolidate all the data in a single repository such as S3.

Thanks.

Upvotes: 2

Views: 2560

Answers (2)

Tomek Klas
Tomek Klas

Reputation: 788

You might want to consider using AWS Database Migration Service to continuously sync data between RDS and S3: https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-dms-target/

saying this, it is only sensible when you don't have a read-only replica of the data and the queries might affect source RDS performance.

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 269520

There is no capability to "export RDS to S3 in real time".

However, Amazon Athena can query Amazon RDS databases, so you could have some of your data in Amazon S3 and some in Amazon RDS.

See: Query any data source with Amazon Athena’s new federated query | AWS Big Data Blog

What you are describing sounds like a data warehouse, where information is extracted from many information sources and is stored in one place for easy querying -- often in 'wide' tables to make querying simpler. However, this is very difficult to do "in real time". It is typically updated nightly, or perhaps hourly.

Upvotes: 2

Related Questions