Heroku Postgres Connection Limit?

Question

I'm building a website attached to a Heroku Postgres database and am using the free hobby dev plan. Per Heroku, this means there's a "Maximum of 20 connections." Does this mean that a maximum of 20 people can be using the website with data being collected by the database on the back end? Any idea what happens if connections go above that level? The paid plans go up to a maximum connection limit of 500, but even that seems low to me if people are using this at the enterprise level. Any color on this would be greatly appreciated. There was a prior question on this but the answer wasn't quite clear to me.

Thanks!

What does database connection limit mean?

Answer 1

PostgreSQL could be configured to limit the number of simultaneous connections to the database. The Heroku comes with plans having connection limits. The 'Hobby' plans come with 20 connections whereas standard plans comes starting with 120 connections. When we start developing and testing, especially automated testings, the hobby plans raise the error PG::Error (FATAL: too many connections for role "xxxxxxx"). If we check the connections with Heroku CLI, we get

Heroku CLI

The immediate solution is to kill all connections with the command :

$ heroku pg:killall --app <app name>

This is not a permanent solution. We had the same issue with this website also. We tried many solutions available in the internet, especially in stack overflow.

It is very important to know how to calculate the no of connections required. Heroku documentation says...

Assuming that you are not manually creating threads in your application code, you can use your web server settings to guide the number of connections that you need. The Unicorn web server scales out using multiple processes, if you aren’t opening any new threads in your application, each process will take up 1 connection. So in your unicorn config file if you have worker_processes set to 3 like this:

worker_processes 3 Then your app will use 3 connections for workers. This means each dyno will require 3 connections. If you’re on a “Dev” plan, you can scale out to 6 dynos which will mean 18 active database connections, out of a maximum of 20. However, it is possible for a connection to get into a bad or unknown state.

Solution - Limit connections with PgBouncer

The easiest fix is to limit the connections with PG bouncer. For many frameworks, you must disable prepared statements in order to use PgBouncer. Then add the PgBouncer buildpack to your app.

$ heroku buildpacks:add https://github.com/heroku/heroku-buildpack-pgbouncer

The output will be something like

Buildpack added. Next release on will use:

1. heroku/python
2. https://github.com/heroku/heroku-buildpack-pgbouncer Run git push heroku master to create a new release using these buildpacks. Now you must modify your Procfile to start PgBouncer. In your Procfile add the command bin/start-pgbouncer-stunnel to the beginning of your web entry. So if your Procfile was

web: gunicorn .wsgi:application --worker-class gevent Change it to:

web: bin/start-pgbouncer-stunnel gunicorn .wsgi:application --worker-class gevent Commit the results to git, test on a staging app, and then deploy to production.

On deployment, you will see

OUTPUT

Answer 2

Depending on the web-framework you are using this can be different, but:

Typically you will have a maximum of one database connection per server process. This could be one per running web- or worker-dyno. Or more if your framework runs multiple thread / worker processes per dyno (most do).

These connections are then only used if there is an actual request to your application, not when the use is just viewing a page.

When you're running an async framework (node.js for example, or greenlets in python) this get's a little more complicated.

The easy way: just test it. You'll see the current connection count in the heroku interfaces. There are frameworks and services in the wild that let you test concurrent users.

The even easier way (since this runs on hobby plans, it seems like a hobby application): just see when it breaks :) .