CODEWITHSUNDEEP
spring-bootjava-8datasourcejdbctemplate
Alex Man
Alex Man

Reputation: 4886

Getting too many connection for role when using DataSource

I have a Rest service and when it gets it has to do some insertion and updation to almost 25 database. So when I tried like the below code, it was working in my localhost but when I deploy to my staging server I was getting FATAL: too many connections for role "user123"

List<String> databaseUrls = null;
databaseUrls.forEach( databaseUrl -> {
    DataSource dataSource = DataSourceBuilder.create()
            .driverClassName("org.postgresql.Driver")
            .url(databaseUrl)
            .username("user123")
            .password("some-password")
            .build();
    JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
    jdbcTemplate.update("Some...Update...Query");
});

As per my understanding DataSource need not to be closed because it is never opened.

Note:

A DataSource implementation need not be closed, because it is never “opened”. A DataSource is not a resource, is not connected to the database, so it is not holding networking connections nor resources on the database server. A DataSource is simply information needed when making a connection to the database, with the database server's network name or address, the user name, user password, and various options you want specified when a connection is eventually made.

Can someone tell why I am getting this issue

Upvotes: 1

Views: 1155

Answers (2)

slesh
slesh

Reputation: 2017

The problem is in DataSourceBuilder, it actually creates of the connection pools which spawns some number of connections and keeps them running:

private static final String[] DATA_SOURCE_TYPE_NAMES = new String[] {
            "org.apache.tomcat.jdbc.pool.DataSource",
            "com.zaxxer.hikari.HikariDataSource",
            "org.apache.commons.dbcp.BasicDataSource" };

Javadoc says:

/**
 * Convenience class for building a {@link DataSource} with common implementations and
 * properties. If Tomcat, HikariCP or Commons DBCP are on the classpath one of them will
 * be selected (in that order with Tomcat first). In the interest of a uniform interface,
 * and so that there can be a fallback to an embedded database if one can be detected on
 * the classpath, only a small set of common configuration properties are supported. To
 * inject additional properties into the result you can downcast it, or use
 * <code>@ConfigurationProperties</code>.
 */

Try to use e.g. SingleConnectionDataSource, then your problem will gone:

List<String> databaseUrls = null;
Class.forName("org.postgresql.Driver");
databaseUrls.forEach( databaseUrl -> {
    SingleConnectionDataSource dataSource;
    try {
        dataSource = new SingleConnectionDataSource(
                databaseUrl, "user123", "some-password", true /*suppressClose*/);
        JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
        jdbcTemplate.update("Some...Update...Query");
    } catch (Exception e) {
        log.error("Failed to run queries for {}", databaseUrl, e);
    } finally {
        // release resources
        if (dataSource != null) {
            dataSource.destroy();
        }
    }
});

Upvotes: 2

SauriBabu
SauriBabu

Reputation: 434

First thing it is very bad architecture decision to have single application managing 50 database. Anyway instead of creating DataSource in for loop, you should make use of Factory Design pattern to create DataSource for each DB. You should add some connection pooling mechanism to your system . HijariCP and TomcatPool are most widely used. Analyse logs of failure thread for any further issues.

Upvotes: 1

Related Questions