NetworkStream is reading data that should not be there

Question

I have a problem with the NetworkStream reading data from the socket buffer that should not be there. I am sending very big buffers by the way. Right now I have just been testing on the localhost.

Here is how I read data, the first 4 bytes contain the length of the message, then I just read in 4096 chunks until it gets to the length of the message.

    protected TcpClient tcpObject;
    protected NetworkStream tcpStream;

    private void HandleComm()
    {
        try
        {
            tcpStream = tcpObject.GetStream();
            byte[] totalByteAray = new byte[constIntSize];
            byte[] message = new byte[constChunkSize];
            byte[] fullMessage = new byte[0];

            //this is how many bytes long the message will be
            int totalBytes = 0;
            int currentBytes = 0;
            int chunkSize = constChunkSize;

            while (true)
            {
                //skip reading if no data is available
                //DataAvailable does not tell you when all the data has arrived
                //it just tell you if some data has arrived
                if (tcpStream.CanRead)
                {
                    totalBytes = 0;
                    currentBytes = 0;
                    message = new byte[constChunkSize];
                    chunkSize = constChunkSize;

                    //The first 4 bytes of the message will always contain the length of the message, not including
                    //the first 4 bytes. This is how you know when to stop reading.
                    tcpStream.Read(totalByteAray, 0, constIntSize);                        
                    //there are 4 bytes in a 32 bit number, so totalByteArrayContains 4 index that is a byte which is
                    //the 32 bit int that tells us how many bytes the whole message will be.
                    //now convert the totalByteArray to a 32bit int
                    totalBytes = BitConverter.ToInt32(totalByteAray, 0);
                    Console.WriteLine("reading " + totalBytes);
                    //fullMessage will contain the entire message but it has to be built message by message.                    
                    fullMessage = new byte[totalBytes];
                    //keep reading until we get all the data
                    while (currentBytes < totalBytes)
                    {

                        //when you send something over TCP it will some times get split up
                        //this is why you only read in chuncks, 4096 is a safe amount of bytes
                        //to split the data into.
                        if (totalBytes - currentBytes < constChunkSize)
                        {
                            chunkSize = totalBytes - currentBytes;
                            message = new byte[chunkSize];
                        }

                        tcpStream.Read(message, 0, chunkSize);
                        //since we know each chunk will always come in at 4096 bytes if it doesn't that means that it's the end
                        //this part cuts off the extra empty bytes                           

                        //copy the message to fullMessage starting at current bytes and ending with the bytes left
                        message.CopyTo(fullMessage, currentBytes);
                        currentBytes += chunkSize;                            
                    }

                    //message has successfully been received
                    if (totalBytes != 0)
                    {

                        if (OnRawDataReceived != null)
                        {
                            RawDataReceivedArgs args = new RawDataReceivedArgs();
                            args.Data = new byte[fullMessage.Length];
                            fullMessage.CopyTo(args.Data, 0);
                            OnRawDataReceived(this, args);
                        }

                        totalBytes = 0;
                    }
                }
            }
        }
        catch
        {
            connectionStatus = ConnectionStatus.NotConnected;
            if (OnDisConnect != null)
                OnDisConnect(this, null);
        }
    }

Here is how I am sending the data, I just get the length of the message and then create a new message with the first 4 bytes being the length of the message and the rest being the actual message.

    protected void sendData(byte[] data)
    {
        //we need to know how big the data that we are sending will be
        int length = data.Length;
        System.Console.WriteLine("writing " + length);
        //convert the 32bit int to a 4 byte array
        byte[] lengthArray = BitConverter.GetBytes(length);

        //init the main byte array that will be sent over
        byte[] buffer = new byte[length + constIntSize];

        //the first 4 bytes will contain the length of the data
        lengthArray.CopyTo(buffer, 0);

        //the rest of the buffer will contain the data being sent
        data.CopyTo(buffer, constIntSize);

        //wite it to the client stream
        tcpStream.Write(buffer, 0, length + constIntSize);
        //now send it
        tcpStream.Flush();           
    }

For some reason I am getting reading data the shouldn't be on the buffer. Here is the console output.

server ------------- client

writing 1024 -> reading 1024

reading 1228800 <- writing 1228800

writing 1024 -> reading 1024

reading 1228800 <- writing 1228800

reading 7224842

So when I click a button it sends a request saying I want a image from a web camera, the request is the 1024 bytes. The client reads it and sends the image which is the 1228800 bytes. The first time I do this, it always works. The second time I clicked it, the client sent back the 1228800 bytes, the server read the correct number of bytes and then found more bytes to read when the socket buffer should have been empty. I did not have 7224842 bytes in the socket buffer, that is just what the first 4 bytes of the read said.

Any ideas of why the buffer is getting extra data in it? Everything seems to work well when I send smaller messages, but this is driving me crazy.

Answer 1

tcpStream.Read(totalByteAray, 0, constIntSize);
...
tcpStream.Read(message, 0, chunkSize);

and there we have the entire problem. It is a requirement that you check the return to this. It is not guaranteed (and for network-based IO, pretty unlikely) that you will get the entire buffer all at once - packets come in as-and-when, and the API will give you what it can. Rather, you will get "some" (result > 0 and <= count) or "none" (result <= 0).

If you want to read exactly that much data, then write a utility method:

static void ReadExact(Stream stream, byte[] buffer, int offset, int count)
{
    int read;
    while(count > 0 && (read = stream.Read(buffer, offset, count)) > 0) {
        offset += read;
        count -= read;
    }
    if(count != 0) throw new EndOfStreamException();
}