How to properly separate packets using Sockets stream? C#

Question

I'm building a server / client application and I'm looking through options for separating packets. I've read that the most proper one would be creating a header that contains information on how big the payload is and then read until it ends.

How does that programmatically works?

Also separating those by using "\n" newline. A proper example would be nice.

I'm async receiving data this way:

private void AsyncReceive(IAsyncResult result)
    {
        int bytesTransfered;

        try
        {
            bytesTransfered = _handle.EndReceive(result);

            if(bytesTransfered <= 0)
            {
                throw new Exception("No bytes transfered");
            }
        }
        catch(NullReferenceException)
        {
            return;
        }
        catch(ObjectDisposedException)
        {
            return;
        }
        catch(Exception)
        {
            return;
        }


        byte[] received = new byte[bytesTransfered];

        try
        {
            Array.Copy(_readBuffer, received, received.Length);
        }
        catch(Exception)
        {
            Disconnect();
            return;
        }


        // How should I process the received data now?


        try
        {
            _handle.BeginReceive(_readBuffer, 0, _readBuffer.Length, SocketFlags.None, AsyncReceive, null);
        }
        catch(ObjectDisposedException)
        {
            return;
        }
        catch(Exception)
        {

        }
    }

Answer 1

First you need to distinguish between different types of messages. You can use single byte for that, which will allow for up to 255 different message types. Make an enum for that, and an attribute to mark your messages (see below):

enum MessageType : byte {
    FirstMessage,
    SecondMessage
}

class MessageAttribute : Attribute {
    public MessageAttribute(MessageType type) {
        Type = type;
    }

    public MessageType Type { get; private set; }
}

Second, you need compact serializer for your messages. One good option is protobuf - it it's very compact (does not serialize property names, only values and so on) while still easy to use.

[Message(MessageType.FirstMessage)]
[ProtoContract]
class MyFirstMessage {
    [ProtoMember(1)]
    public string Value { get; set; }
    [ProtoMember(2)]
    public int AnotherValue { get; set; }
}

[Message(MessageType.SecondMessage)]
[ProtoContract]
class MySecondMessage {
    [ProtoMember(1)]
    public decimal Stuff { get; set; }
}

Third you need to know the length of a message, as caller says you. Use 2 or 4 bytes for that (size of Int16 and Int32 types respectively).

So our format would be: 1 byte - message type. 2-5 bytes - message size, 5-5+size bytes - protobuf serialized message. Then read your stream in three steps, as defined below:

class MessageReader {
    static readonly Dictionary<byte, Type> _typesMap = new Dictionary<byte, Type>(); 
    static MessageReader() {
        // initialize your map
        // this is executed only once per lifetime of your app
        foreach (var type in Assembly.GetExecutingAssembly().GetTypes().Where(c => c.GetCustomAttribute<MessageAttribute>() != null)) {
            var message = type.GetCustomAttribute<MessageAttribute>();
            _typesMap.Add((byte)message.Type, type);
        }
    }

    public async Task<object> Read(Stream stream) {
        // this is your network or any other stream you have
        // read first byte - that is message type
        var firstBuf = new byte[1];
        if (await stream.ReadAsync(firstBuf, 0, 1) != 1) {
            // failed to read - end of stream
            return null;
        }

        var type = firstBuf[0];
        if (!_typesMap.ContainsKey(type)) {
            // unknown message, handle somehow
            return null;
        }
        // read next 4 bytes - length of a message
        var lengthBuf = new byte[4];
        if (await stream.ReadAsync(lengthBuf, 0, 4) != 4) {
            // read less than expected - EOF
            return null;
        }
        var length = BitConverter.ToInt32(lengthBuf, 0);
        // check if length is not too big here! or use 2 bytes for length if your messages allow that
        if (length > 1*1024*1024) {
            // for example - adjust to your needs
            return null;
        }
        var messageBuf = new byte[length];
        if (await stream.ReadAsync(messageBuf, 0, length) != length) {
            // didn't read full message - EOF
            return null;
        }
        try {
            return ProtoBuf.Serializer.NonGeneric.Deserialize(_typesMap[type], new MemoryStream(messageBuf));
        }
        catch {
            // handle invalid message somehow
            return null;
        }
    }
}

After you read one message from stream - continue in the same way to read next message. Read calls will block until new data arrives. If there is any violation of protocol - drop connection.

Answer 2

Have you not considered using a TCPClient and TCPListener, and then a NetworkStream? Sockets are pretty low level and probably not needed in the majority of cases.

See this post: How reading messages from server?(TCP)

Also, do not catch exceptions that you cannot recover from, unless you log and rethrow them. This will cause very hard to debug behaviors when exceptions get silently swallowed up.