CODEWITHSUNDEEP
apache-flinkflink-streamingjedis
avidlearner
avidlearner

Reputation: 253

Redis connection reset in Flink

I have a class that extends a RichFlatmapFunction in my Flink stream job. I create a Jedis instance in the open() method and close it (jedis.close()) in the close() method, so that all the records that pass through the transformation use the same Jedis instance. This approach has not given me any connection errors before. But in a recent job run, I got the error,

"redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Connection reset".

Could this be because I am using the same jedis instance (which remains open) for all the records? I have not set any timeout while creating the instance. So the timeout is also the default value.

Would I be able to avoid this error if I use JedisPool so as to retrieve and close Jedis instances for each record?

Upvotes: 0

Views: 970

Answers (1)

Aarish Ramesh
Aarish Ramesh

Reputation: 7043

Connection reset happens where there is any disruption between server and client connection at the network level i.e any Network lost,firewall or application crash or intended close

Also there is a possibility of intended close because of the idle timeout.

Now coming to using JedisPool, Using connection pool is the best practice in multithreaded environments as reusing connections is efficient. So please try to use them with proper configuration

Upvotes: 1

Related Questions