CODEWITHSUNDEEP
javascriptnode.jssockets
majidarif
majidarif

Reputation: 20025

Writing to socket very fast causes joined buffers

I have this code to test out the server,

var client = net.connect({port: 3000}, onConnected);

client.setNoDelay(false);

function onConnected() {
    while (true) {
        client.write('hello');
    }
}

using the debug module, I see 0ms for each request which is expected considering it is on the same machine.

The server should receive <Buffer 68 65 6c 6c 6f> but the server actually receives:

<Buffer 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6^Cc 6c 6f 68 65 6c 6c 6f 68 ...>

Odd behaviour? Actually not, if I understand this correctly, this is happening because the write isn't flushed immediately causing it to be joined before actually getting sent.

But problem is how do I get around this and actually receive the right bytes even if the writing to the socket is very fast.

An idea is to have all the written data to be prepended with the size. Then read from that size and cut the buffer from there. But question is how do I do that?

I don't think I can do that with something like this:

socket.on('data', function(data) {
  var len = data[0];

  // hmmmm
})

So,

Upvotes: 2

Views: 343

Answers (2)

majidarif
majidarif

Reputation: 20025

@6502 answer was great but it was not really for my use case.

Warning: I'm sharing the module I wrote to help me with my question here.

So, I saw this module called carrier but it only worked for strings, or new line separated data. What I needed was for a Buffer, so based on the carrier I wrote StreamFrame.

Boundr is a simple implementation of taking the length of the data which is on the first 2 bytes and just cut them there, thus boundr for boundaries. I wrote a blog post about it found here.

Thanks to @vaultah comment which helped me to find carrier.

Upvotes: 0

6502
6502

Reputation: 114539

TCP sockets implement stream communication, not packets. This means you can write 3 bytes followed by 3 bytes and the receiver can get instead 5 bytes on first read followed by 1 byte on second read.

If you want to send a packet you need to encode it down the stream yourself and the standard way is to send first the packet size and then the packet itself. E.g.

var szbuf = Buffer(2); // max packet size is 65535
szbuf[0] = msg.size & 255;
szbuf[1] = msg.size >> 8;
write(szbuf); // Send the size
write(msg);   // Send the message

The reader will need first to read 2 bytes only, compute the size and then read that amount of bytes to get the message.

If you need to use message-based communication may be using an higher level library like 0MQ is a better idea.

If instead you're only writing the server and the protocol doesn't contain already a message boundary (e.g. a newline for a ascii telnet-like protocol) then it's hard (impossible) to provide reliable message passing because writing and reading boundaries are not respected by TCP and delay over the internet can be arbitrary.

This means that the client has no way to know if the message is complete or not.

Upvotes: 4

Related Questions