Make cluster of netty-socketio server

Question

I am using This for socketio setup. I have 2 different socketio server(let say server1 and server2) running in cluster with using of RedissonStoreFactory

My issue is, If any client connects with server1, then server2 has no information if connected client. ie. If 2 clients connect with server1 and If I execute server.getAllClients() on server2 it returns the empty list instead of the list with 2 counts.

Here is my code running on 2 different machines.

@SpringBootApplication
public class Application {


    private static final Logger LOGGER = Logger.getLogger(Application.class);

    @Value("${test.socketio.hostName}")
    private String socketIOHostName;

    @Value("${test.socketio.port}")
    private Integer socketIOport;

    @Value("${test.dedisson.redissonAddress}")
    private String redissonAddress;

    @Autowired
    private RedissonClient redissonClient;

    @Bean
    public SocketIOServer socketIOServer() {
        LOGGER.info("Socket server starting on host=" + socketIOHostName + ", port=" + socketIOport);
        Configuration config = new Configuration();
        config.setHostname(socketIOHostName);
        config.setPort(socketIOport);

        StoreFactory redissonStoreFactory = new RedissonStoreFactory(redissonClient);
        config.setStoreFactory(redissonStoreFactory);
        SocketIOServer server = new SocketIOServer(config);
        server.start();
        LOGGER.info( "Socket server started");
        return server;
    }

    /**
     * 
     * @return
     */
    @Bean
    public RedissonClient getRedissonClient(){
        LOGGER.info("creatting redisson client on redissonAddress="+redissonAddress);
        Config config = new Config();
        config.useSingleServer().setAddress(redissonAddress);
        RedissonClient redisson = Redisson.create(config);
        LOGGER.info("redisson client connected");
        return redisson;
    }


    public Application() {
        //Nothing to be done here
    }

    public static void main(String[] args) {
        ConfigurableApplicationContext context = SpringApplication.run(Application.class, args);
    }

    @Bean
    public SpringAnnotationScanner springAnnotationScanner(SocketIOServer ssrv) {
        return new SpringAnnotationScanner(ssrv);
    }
}

I prefer 2 instances for failover condition.If server1 is down then server2 will send the notification to connected clients, but in my case, server2 has no idea of the connected client with server1.

Answer 1

I've had a similar problem and solved that with my "self-protocol".

There was a server A connected with client X, and server B connected with client Y. I needed to send a private message from server A to client Y. The solution was made by mapping, in each server, a list of all server hosts, and for every server host we create a "internal client".

Each internal client is responsible for notifying its respective server, then the server could check if client-target exists on its self-list of clients.

Given a server A, if you want to list all clients of a cluster, you can ask for each internal client to send a packet e.g. "REQUEST" for its connection server, then create a listener for "RESPONSE" message and add result to a global list.

build.gradle:

implementation group: 'io.socket', name: 'socket.io-client', version: '1.0.0'

Server

Configuration config = new Configuration();
config.setHostname(LOCALHOST);
config.setPort(PORT);
config.setTransports(Transport.WEBSOCKET); // DONT FORGET THIS LINE !!!

Config redissonConfig = new Config();
redissonConfig.useSingleServer().setAddress("redis://192.168.0.24:6379")
            .setPassword("myPass");
Redisson redisson = (Redisson) Redisson.create(redissonConfig);
RedissonStoreFactory redisStoreFactory = new RedissonStoreFactory(redisson);
config.setStoreFactory(redisStoreFactory);

SocketIOServer server = new SocketIOServer(config);

server.addEventListener("REQUEST", Object.class, (client, data, ackSender) -> {
    server.getBroadcastOperations().sendEvent("RESPONSE", server.getAllClients());
});

Internal Clients

List<SocketIOClient> allClients = new ArrayList<>();
List<Socket> internalClients = new ArrayList<>();
String[] hostnames = { "http://localhost:8081", "http://localhost:8082" };

for (String hostname : hostnames) {
    IO.Options opts = new IO.Options();
    opts.transports = new String[] { WebSocket.NAME };  // DONT FORGET THIS LINE !!!
    Socket socket = IO.socket(hostname, opts);

    socket.on("RESPONSE", args -> {
        List<SocketIOClient> currentList = (List<SocketIOClient>) args[0];
        allClients.addAll(currentList);
    });

    socket.connect();
    internalClients.add(socket);
}

for (Socket socket : internalClients) {
    socket.emit("REQUEST", "foo"); // THIS LINE WILL FILL CLIENTS LIST, VIA CALLBACK, FOR EACH SERVER
}

Answer 2

Though it's too late, this reference implementation may help others:

https://github.com/wuxudong/socket.io-netty-redisson/blob/master/server/src/main/java/com/corundumstudio/socketio/demo/ChatLauncher.java

Answer 3

If 2 clients connect with server1 and If I execute server.getAllClients() on server2 it returns the empty list instead of the list with 2 counts.

This is expected. Server2 might have some information about connected clients with server1 (as they use redis with pub/sub) like sessionId (look into class BaseStoreFactory)and etc but server2 is not connected with those clients, there is no Channel present between server2 and clients connected with server1. Give it a thought, on low level, when no tcp socket connection is present then how come server2 can communicate with client.

I prefer 2 instances for failover condition.If server1 is down then server2 will send the notification to connected clients, but in my case, server2 has no idea of the connected client with server1.

Use Nginx (with iphash)or any proxy on top of your cluster, as soon as one server goes down, the client will try to reconnect and nginx will redirect it to the other server. From clinet point of view, there won't be much delay in that.