Cannot get Jedis connection, Could not get a resource from the pool

Question

I am creating a Web Application and using Redis for data storage. I have used JedisPoolConfig (redis client-jedis version 2.9.0) . So when I test my application with help of Apache Jmeter with following configurations :

Number of threads(Users): 30
Ramp-Up period(in seconds) : 0
Loop-Count : 1

and JedisPoolConfig with following configurations :

redis.sentinel.enable=true
redis.sentinel.ip=localhost
redis.sentinel.port=6379
redis.instance.account.masterName=master
redis.pool.max-active = 20
redis.pool.max-idle = 10

with this code (which is created as Bean on startup of Spring Application) :

JedisConnectionFactory factory = new JedisConnectionFactory();
        factory.setPort(port);
        factory.setHostName(hostName);
        factory.afterPropertiesSet();
        factory.setUsePool(true);
        JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
        jedisPoolConfig.setMaxTotal(redisPoolMaxActive);
        jedisPoolConfig.setMaxIdle(redisPoolMaxIdle);
        factory.setPoolConfig(jedisPoolConfig);
        return factory;

with this code for fetching new Connection everytime a Request comes :

@Autowired
private JedisConnectionFactory redisConnectionFactory;
private StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
private RedisTemplate getNewConnection(){
        RedisTemplate redisTemplate = new RedisTemplate();
        redisTemplate.setConnectionFactory(redisConnectionFactory);
        redisTemplate.setKeySerializer(stringRedisSerializer);
        redisTemplate.setHashKeySerializer(stringRedisSerializer);
        redisTemplate.setValueSerializer(stringRedisSerializer);
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }

Edit : my usage for redisTemplate for multi/exec inside Redis :

redisTemplate.setEnableTransactionSupport(true);

redisTemplate.multi();

for (ArrayList<Pair<String, String>> pairs : keys){
    for (Pair<String,String> pair: pairs)
        redisTemplate.opsForHash().get(makeMap(pair.getFirst(), version) , pair.getSecond());
}

return redisTemplate.exec();

I successfully get the response for 20 requests but could not get the response for rest 10 requests. The issue I am facing is after 20 connections have been used by first 20 requests , the rest 10 requests are not able to get the jedis connection from the pool and thus I get the following exception :

    2017-10-06 17:38:11.753 ERROR 3539 --- [io-8080-exec-12] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisException: Could not get a resource from the pool] with root cause

java.util.NoSuchElementException: Timeout waiting for idle object
    at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:449) ~[commons-pool2-2.4.2.jar:2.4.2]
    at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) ~[commons-pool2-2.4.2.jar:2.4.2]
    at redis.clients.util.Pool.getResource(Pool.java:49) ~[jedis-2.9.0.jar:na]
    at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226) ~[jedis-2.9.0.jar:na]
    at redis.clients.jedis.JedisPool.getResource(JedisPool.java:16) ~[jedis-2.9.0.jar:na]
    at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:194) ~[spring-data-redis-1.8.6.RELEASE.jar:na]
    at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:348) ~[spring-data-redis-1.8.6.RELEASE.jar:na]
    at org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:129) ~[spring-data-redis-1.8.6.RELEASE.jar:na]
    at org.springframework.data.redis.core.RedisConnectionUtils.bindConnection(RedisConnectionUtils.java:67) ~[spring-data-redis-1.8.6.RELEASE.jar:na]
    at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:192) ~[spring-data-redis-1.8.6.RELEASE.jar:na]
    at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:169) ~[spring-data-redis-1.8.6.RELEASE.jar:na]
    at org.springframework.data.redis.core.RedisTemplate.multi(RedisTemplate.java:868) ~[spring-data-redis-1.8.6.RELEASE.jar:na]
    at net.media.mml.redis.repository.RedisRepository.makeTransaction(RedisRepository.java:50) ~[max-data-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
    at net.media.mml.redis.repository.RedisRepository.getFromRedis(RedisRepository.java:66) ~[max-data-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
    at net.media.mml.service.DimensionGetter.getDimensions(DimensionGetter.java:64) ~[ml-api.jar:0.0.1-SNAPSHOT]
    at net.media.mml.service.MLQueriesGetter.makeQueries(MLQueriesGetter.java:32) ~[ml-api.jar:0.0.1-SNAPSHOT]
    at net.media.mml.controller.MaxQueryController.query(MaxQueryController.java:61) ~[ml-api.jar:0.0.1-SNAPSHOT]
    at sun.reflect.GeneratedMethodAccessor270.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.10.RELEASE.jar:4.3.10.RELEASE]

I have searched over the net but not able to find the solution to this. Can anybody help me?

Answer 1

Using Redis transactions requires invocations in a callback. setEnableTransactionSupport(true) is intended for transaction-manager/@Transactional use. See the reference documentation for further details.

Do not use setEnableTransactionSupport(true) unless you have a transaction manager in place.

You're code should look like:

List<Object> txResults = redisTemplate.execute(new SessionCallback<List<Object>>() {
  public List<Object> execute(RedisOperations operations) throws DataAccessException {
    operations.multi();
    operations.opsForSet().add("key", "value1");

    // This will contain the results of all ops in the transaction
    return operations.exec();
  }
});