CODEWITHSUNDEEP
sqlpostgresqlamazon-web-servicesamazon-rdspostgresql-10
user1393608
user1393608

Reputation: 1349

AWS RDS Postgres maximum instance and table size

Thanks in advance!

I am planning to use AWS RDS Postgres for pretty big data (> ~50TB) , but I have couple of questions un-answered

  1. Is 16TB the maximum limit for AWS RDS Postgres instance, if so how do people store > 16TB data.

  2. Is the limit of 16TB for RDS the maximum database size post compression that Postgres can store on AWS.

  3. Also I do not see any option for enabling compression while setting up AWS RDS Postgres DB instance. How to enable compression in AWS RDS Postgres?

I have followed :

  1. https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html

  2. https://blog.2ndquadrant.com/postgresql-maximum-table-size/ (wherein Postgres table can have size greater than 32TB).

  3. https://wiki.postgresql.org/wiki/FAQ#What_is_the_maximum_size_for_a_row.2C_a_table.2C_and_a_database.3F

Upvotes: 4

Views: 9182

Answers (3)

sumitskumar
sumitskumar

Reputation: 31

RDS limit has been revised to 64 TB.

MariaDB, MySQL, Oracle, and PostgreSQL database instances: 20 GiB–64 TiB

SQL Server for Enterprise, Standard, Web, and Express editions: 20 GiB–16 TiB

Upvotes: 3

Kevin J
Kevin J

Reputation: 91

In addition to RDS for PostgreSQL, which has a 32 TiB limit, you should take a look at Amazon Aurora PostgreSQL, which has a 64 TiB limit. In both cases, the largest single table you can create is 32 TiB, though you can't quite reach that size in RDS for PostgreSQL as some of the space will be taken up by the system catalog.

Full disclosure: I am the product manager for Aurora PostgreSQL at AWS.

Upvotes: 5

Matt
Matt

Reputation: 987

As of 2019/02/11, Amazon's docs state that the maximum database size for a Postgres RDS instance is 32TiB. Additionally, this 32TiB limit appears to be a hard limit (Some AWS limits are considered 'soft' in that a user can request that they be raised)

As others have suggested, one option would be to manage your own database, however this is not likely to be easy given the scale of data we are talking about here.

Staying within the AWS ecosystem, another option would be to store all your data in S3, and use AWS Athena to run queries, although depending on what sort of queries you'll be running, it could get quite expensive.

As for your question about compression, if you are storing data in s3, you can compress it before you upload. You might also find this answer helpful.

Upvotes: 3

Related Questions