CODEWITHSUNDEEP
javaredisvert.x
0x4b50
0x4b50

Reputation: 679

Subscription to Redis Stream Blocks Connections for Other Requests With Vert.x

I'm implementing an application with Vert.x (and their Redis Client library) that is able to listen to Redis Streams. For that I have n instances in my app subscribing to it. The Vert.x documentation states:

A common configuration is to set the maximum size of the pool to the number of available CPU cores

The pool provides connections to be used for requests to Redis. For instance, the pool size could to 6. If more than 6 instances trying to subscribe, the first 6 have no problems, but all the instances trying to subscribe afterwards can't because no connections are left.

Is the "common configuration" are hard limit? Could I also have 10k connections?

Does Vert.x provide a way to separate pools for different purposes (for subscriptions and for other stuff)?

Is there a solution to handle many connections mainly used for stream subscriptions with Vert.x?

Unfortunately, the Vert.x documentation doesn't provide much information.

Upvotes: 0

Views: 287

Answers (1)

Paulo Lopes
Paulo Lopes

Reputation: 5801

Vert.x redis client will attempt to work with the least number of connections to redis as this usually gives better performance for both vert.x and redis.

When working with simple command and response, for example, SET key value then a pool has benefits as it reuses connections and the whole flow is faster. However, for pub/sub, it is best if you use a dedicated connection (not a pool)

Redis.createClient(
  vertx,
  "redis://localhost:6379")
  .connect()
  .onSuccess(conn -> {
    // use the connection to subscribe...
  });

If you wish to use a pool for subscriptions (which doesn't give any advantage in terms of performance) you still can, but need to tweak the defaults to allow more connections than usual. You'll do this with RedisOptions object:

new RedisOptions()
    .setConnectionString("redis://localhost:7006")
    // allow at max 128 connections to redis
    .setMaxPoolSize(128)
    // allow 512 connection requests to queue waiting
    // for a connection to be available.
    .setMaxWaitingHandlers(512))

This means that there will be 128 connections readily available and for commands that go over the connections in use, the client will queue them up to 512 until a connection is available, otherwise starts failing commands. Usually, this prevents your server to overload Redis or run out of memory/sockets.

Upvotes: 1

Related Questions