CODEWITHSUNDEEP
postgresqlamazon-web-servicesautoscalingrdspgbouncer
najeeb
najeeb

Reputation: 813

Autoscaling limited by RDS connection

I have some nightly jobs that are running on EC2 and the number of machines is scaled by the number of messages in SQS. My process requires reads from a Postgres RDS database. Now these are the issues I am facing.

  1. Not able to scale beyond a certain number because of the unavailability of connections.
  2. I tried creating a connection pool using pgbouncer, and tried with different settings as well, but it's missing a lot of data on the resultant set.

Upvotes: 0

Views: 1178

Answers (1)

Vorsprung
Vorsprung

Reputation: 34397

Make your postgresql RDS install multi AZ. Then you can make read replicas on demand and scale read performance with your load.

To answer the comments:

  • Some extra "plumbing" is required to make the connections to the read replica. Maybe route53 dynamically updated records as the scaling happens or something like haproxy
  • The reason I mention multi AZ is that this would help prevent downtime during an auto scaling event bringing up the read replica

It would be simpler (but more costly) to permanently bring up a read replica and use DNS round robin to share the load

See https://aws.amazon.com/blogs/aws/amazon-rds-announcing-read-replicas/ for information on read replicas

Upvotes: 1

Related Questions