C# Asynchronous Socket client/server

Question

I am having troubles programming a TCP client/server using sockets. I've made a little programme that use the MSDN examples i found here : http://msdn.microsoft.com/en-us/library/bew39x2a%28v=vs.110%29.aspx but with a few modifications like a queue on the send function:

public void Send(String data)
{
    // Wait for server connection
    if (connectDone.WaitOne())
    {
        // If already sending data, add to the queue
        // it will be sent by SendCallback method
        if (_sending)
        {
            _queue.Enqueue(data);
        }
        else
        {
            // Convert the string data to byte data using ASCII encoding.
            byte[] byteData = Encoding.ASCII.GetBytes(data);

            // Begin sending the data to the remote device.
            _sending = true;
            SocketHolder.BeginSend(byteData, 0, byteData.Length, 0, SendCallback, SocketHolder);
        }
    }
}

The data are read by the server via the readcallback function :

    public void ReadCallback(IAsyncResult ar)
    {
        string content = string.Empty;

        // Retrieve the state object and the handler socket
        // from the asynchronous state object.
        StateObject state = (StateObject)ar.AsyncState;
        Socket handler = state.workSocket;

        // Read data from the client socket. 
        int bytesRead = handler.EndReceive(ar);

        if (bytesRead > 0)
        {
            // There  might be more data, so store the data received so far.
            state.sb.Append(Encoding.ASCII.GetString(state.buffer, 0, bytesRead));

            // Check for end of message tag & raise event
            content = state.sb.ToString();
            if (content.IndexOf("</MetrixData>") > -1)
            {
                Console.WriteLine("Read {0} bytes from socket. \n Data : {1}", content.Length, content);
                OnMessageReceived(EventArgs.Empty, content);
                state.sb = new StringBuilder(); // Clear the message from state object string builder
            }
            handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0, ReadCallback, state);

        }
    }

Well, now here is my problem : the whole point of my application is to send variable to a server at high frequency (up to 60 time per seconds for each variable). BUT, when i try to update very quickly my data, it seems the string builder of my state object doesn't have the time to clear properly as i reiceive multiple data at the same time within my OnMessageReceived function (which is a problem because i serialize every data i send so i end up with multiple root element in the message received by the server).

The whole project is available here if you want to have a closer look (not really sure my explanations are clear...)

Thank you in advance for your help & your time :)

---

EDIT : Sorry, i will try to give a beter explanation of my problem :p

Here is a message example sent & received correctly when i update a data solely.

<MetrixData>
  <DataId>IntTest</DataId>
  <Type>System.Int32</Type>
  <Value TimeStamp="22/07/14 22:22:19">10</Value>
</MetrixData>

And here is what my server receive if i make multiple update of my data in a very short period of time.

<MetrixData>
  <DataId>IntTest</DataId>
  <Type>System.Int32</Type>
  <Value TimeStamp="22/07/14 22:25:06">12</Value>
</MetrixData><MetrixData>
  <DataId>IntTest</DataId>
  <Type>System.Int32</Type>
  <Value TimeStamp="22/07/14 22:25:06">13</Value>
</MetrixData><MetrixData>
  <DataId>IntTest</DataId>
  <Type>System.Int32</Type>
  <Value TimeStamp="22/07/14 22:25:06">14</Value>
</MetrixData>

(can receive 2, 3... or up to 10 messages at the same time)

I can't figure out why my ReadCallback doesn't detect the end of the current message and doesn't reset the buffer which should be done by this sample of ReadCallBack

// Check for end of message tag & raise event
        content = state.sb.ToString();
        if (content.IndexOf("</MetrixData>") > -1)
        {
            Console.WriteLine("Read {0} bytes from socket. \n Data : {1}", content.Length, content);
            OnMessageReceived(EventArgs.Empty, content);
            state.sb = new StringBuilder(); // Clear the message from state object string builder
        }

Answer 1

This is speculation on my part. I think that without packet-orientation (UDP) you just have a continuous TCP stream. You need to implement one of these:

• revert to UDP (maybe using Lidgren) => very expensive and time consuming;
• break up the stream with a separator character => requires a clearly defined set of the alphabet and the separator, example: in BSON (binary JSON) you have the base-64 alphabet and may use any non-base-64 character as a separator suche as '|';
• implement a message envelop that allows you to separate the single messages => this is the natural solution for what you are doing.

The message envelop is as easy as this:

• use a MemoryStream to buffer output data;
• use a BinaryWriter to write into that buffer;
• have a "send/transmit/submit/commit" method that closes the envelop and sends it on the NetworkStream.

This is a quick whip-up:

// NetworkStream ns; <-- already set up

using(MemoryStream memory = new MemoryStream())
using(BinaryWriter writer = new BinaryWriter(memory))
{
    // all output to writer goes here

    // now close the envelop and send it:
    byte[] dataBuffer, sizeBuffer;

    dataBuffer = memory.ToArray();
    sizeBuffer = BitConverter.GetBytes(dataBuffer.Length);
    ns.SendBytes(sizeBuffer, 0, 4); // send message length (32 bit int)
    ns.SendBytes(dataBuffer, 0, dataBuffer.Length); // send message data
}

On the other end of the line you will open the envelop and extract the message with the reverse procedure:

// NetworkStream ns; <-- already set up

byte[] sizeBuffer, dataBuffer;
int size;

sizeBuffer = new byte[4];
ns.ReadBytes(sizeBuffer, 0, 4); // read message length
size = BitConverter.ToInt(sizeBuffer);

dataBuffer = new byte[size];
ns.ReadBytes(dataBuffer, 0, size); // read message data

using(MemoryStream memory = new MemoryStream(dataBuffer))
using(BinaryReader reader = new BinaryReader(memory))
{
    // all input from reader goes here
}

Answer 2

TCP offers you a stream of bytes. There are no messages in TCP. You are probably receiving two logical messages in one read. That causes your processing to be executed once when you wanted it to run twice.

If you want message semantics you have to implement them yourself. Usually, raw TCP connections are a mistake to begin with. Why don't you use a higher-level primitive like HTTP or web services? At the very least, use protopuf as the serialization format. Had you done any of these the problem would not have occurred. Many more landmines are ahead of you when you implement a wire protocol yourself.

Also, your string encoding will fail you once you go outside the ASCII range (in other words, when this app goes into production). Again, this problem only exists because you are doing all this low-level work. Sending messages and strings over a network cable has been automated for you. Utilize that work.