CODEWITHSUNDEEP
node.jssocket.ioload-balancing
Craig
Craig

Reputation: 397

NodeJS/SocketIO messages not being delivered reliably

I am experiencing very strange behaviour with our NodeJS/SocketIO development and I just don't know what is causing my problem. I have looked almost everywhere I can in our code, but cannot find anything obvious that would be causing this issue. In the end I had one of our network guys suggest it may be related to loadbalancing on the NodeJS server. I'm not sure if this is true, but thought I would tell you guys/girls about the problem and see if you have any suggestions.

We run a NodeJS/SocketIO app that basically allows our Web based admin app to monitor connections of users on our websites. Really, all it is supposed to do is receive incoming messages and reformat the message and then rebroadcast them to all clients listening (Which is all our Web based admin's) in real time. Essentially it works like this...

My problem is that when I run an instance of the Web based admin on one PC on one IP, then run another instance of the same Web based admin on another PC and a different IP - both at the same time - They mostly do not see the same traffic. In almost every case, a connection from a user shows in one or the other Web based admin instances, but very rarely, does a connection appear in both as I expect it too. It appears that nearly all messages from the NodeJS/SocketIO are received by one of the Web based admin's but not the other, even though they should all be listening and receiving the same messages. I had designed the code so that when a connection is received by the NodeJS/SocketIO it broadcasts the new connection message to all clients that are listening which should include BOTH Web based admins and both Web based admin instances should see all incoming connections and update their views to match. However, this doesn't happen. The messages are nearly always received by one of the instances of Web based admin and not the other.

Strangely though, when I run 2 instances of the Web based admin on 2 different PC's but on the same IP, the messages are received and processed correctly in both Web based admins. As I would expect if both Web based admins were running on different IP's.

Our network guy suggested it may have to do with load balancing on the NodeJS server. He suggested that maybe when a Web based admin instance connects to the NodeJS server, it is seeing only one instance of the NodeJS server and therefore cannot see all connections that might come in on another instance of the NodeJS server.

The NodeJS/SocketIO app running on the server looks like this...

var app = require('express')();

var http = require('http').Server(app);

var io = require('socket.io')(http);

http.listen(process.env.PORT, function(){
  console.log('listening on PORT:' + process.env.PORT);
});

io.on('connection', function(socket){

// Report a new connection and where it came from
var client_ip_address = socket.handshake.headers['x-forwarded-for'] ||    socket.request.socket.remoteAddress;
console.log("Connection from:" + client_ip_address);

socket.on('disconnect', function () {
    emitMessageToAll({type: "disconnect", ip: client_ip_address, senderID: socket.id});
});

// After a new user connects, we wait for them to emit to us the website they are currently on before notifying all users (Admin)
// Of the new connection. That way Admin can filter users by what site & page they are currently on. 
socket.on('newConnectionInfo',function(data){   
    console.log("User reports connection data: " + data.IP);    
    emitMessageToAll({type: "newConnection", ip: data.IP, url: data.url, senderID: socket.id});
});

socket.on('newBooking',function(data){  
    console.log("User reports new booking: " + data.IP);    
    emitMessageToAll({type: "newBooking", ip: data.IP,PurchDate: data.PurchDate, Surname: data.Surname});
});
});

function emitMessageToAll(data){
    io.emit('message',data);
    console.log("Sending:");
    console.log(data);
}

Anyway, I am completely stumped here as to what might be causing the problem, can you suggest any ideas that might be causing this really strange behaviour ? Is it possibly a load balancing issue or server related issue ?

Upvotes: 0

Views: 141

Answers (1)

Craig
Craig

Reputation: 397

So, the answer in my case was that the server's load balancing was allowing multiple instances of our NodeJS/SocketIO app to run. We had up to 3 instances running concurrently which caused some connections to be routed through instance 1, some through instance 2 and others through instance 3. Once the additional instances of the NodeJS/SocketIO app were closed, the issue resolved itself.

Upvotes: 0

Related Questions