Redis connection lost on Vert.x 3.9.4

Question

After upgrading the vertx-redis-client to version 3.9.4 according to Vert.x documentation

The connection code was needed to be updated to conform the newer version. The code based on the example from the documentation above and looks like this:

Redis.createClient(vertx, redisOptions).connect(onConnect -> {
    if (onConnect.succeeded()) {
        RedisConnection client = onConnect.result();
        RedisAPI redis = RedisAPI.api(client);
        vertx.getOrCreateContext().put("redis", redis);
    }
});

But after an hour (more or less) I'm getting a connection lost with following error stack trace:

io.vertx.redis.client.impl.ConnectionManager lambda$static$0 - Unhandled Error
java.nio.channels.ClosedChannelException 
at io.netty.channel.AbstractChannel$AbstractUnsafe.newClosedChannelException(AbstractChannel.java:957)
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:865) 
at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1367) 
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717) 
at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764) 
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790) 
at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:758) 
at io.vertx.core.net.impl.ConnectionBase.write(ConnectionBase.java:124) 
at io.vertx.core.net.impl.ConnectionBase.writeToChannel(ConnectionBase.java:205) 
at io.vertx.core.net.impl.NetSocketImpl.writeMessage(NetSocketImpl.java:130) 
at io.vertx.core.net.impl.NetSocketImpl.write(NetSocketImpl.java:174) 
at io.vertx.core.net.impl.NetSocketImpl.write(NetSocketImpl.java:168) 
at io.vertx.redis.client.impl.RedisConnectionImpl.lambda$send$1(RedisConnectionImpl.java:136) 
at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:366) 
at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38) 
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) 
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) 
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:497) 
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) 
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 
at java.lang.Thread.run(Thread.java:748)

Does anyone knows why this is happening? Or any idea how to solve this?

Answer 1

If your connection is idle for a long period, Redis server might close the connection. There are couple of ways to resolve this. Something similar is explained in this issue

1. Add fixed periodic dummy command (PING) so the connection doesn't get closed.

   Redis.createClient(vertx, new RedisOptions()).connect(onConnect -> {
       if (onConnect.succeeded()) {
           RedisConnection client = onConnect.result();
           RedisAPI redis = RedisAPI.api(client);
           vertx.getOrCreateContext().put("redis", redis);
   
           vertx.setPeriodic(30 * 1000, handler -> {
               redis.ping(new ArrayList<>(), result -> {
                   if (result.failed()) {
                       System.out.println("Error " + result.cause().getMessage());
                   }
               });
           });
       }
   });
   

2. Add periodic task that will obtain and replace your connection.

   final AtomicReference<RedisConnection> vertxConnection = new AtomicReference<>();
   private void refreshConnection(Redis client, RedisConnection fallback) {
       // get a connection
       client.connect()
         .onFailure(err -> {
           // maybe log, don't change the current connection
           System.out.println("connect failed: " + err.getMessage());
           vertx.setTimer(5000L, t -> refreshConnection(client, fallback));
         })
         .onSuccess(conn -> {
           // swap old with new
           System.out.println("connect swapped: " + (conn != null ? conn.hashCode() : null));
           final RedisConnection old = vertxConnection.getAndSet(conn);
           vertx.setTimer(5000L, t -> refreshConnection(client, old));
           // the fallback isn't needed anymore
           if (fallback != null) {
             fallback.close();
           }
         });
     }
   // create client
   Redis client = Redis.createClient(vertx, redisOptions);
   
   // get a connection
   refreshConnection(client, null);
   

3. Add exception handler to refresh connection when this happens, good example can be found in official docs: https://vertx.io/docs/vertx-redis-client/java/#_implementing_reconnect_on_error