TCP Client - Data gets corrupted only in some parts

Question

I have a client application that sends file to server. The problem is that in client part the file is read correctly, I can save byte array again after I read it and it saves as a valid jpg. On the server end however, the data gets corrupted in some places. All I do is keep adding to the List all incoming data until client closes connection.

I overlapped both client and server data sent/received and as you can see there are places where data doesn't match: http://s16.postimage.org/mtmm1hssl/corrupted_data.png

I don't understand how is this possible, I'm using a big buffer and the image is fairly small, so even when image is received by the server in one 'chunk', the data is still corrupted.

Any ideas why this is happening?

UPDATE:

CLIENT Side:

I'm fetching an image with HttpWebRequest:

                  List<byte> test_buffer = new List<byte>();

                  using (Stream MyResponseStream = hwresponse.GetResponseStream())
                  {
                      byte[] MyBuffer = new byte[4096];
                      int BytesRead;

                      while (0 < (BytesRead = MyResponseStream.Read(MyBuffer, 0, MyBuffer.Length)))
                      {

                          for (int i = 0; i < BytesRead; i++)
                          {
                              test_buffer.Add(MyBuffer[i]); // just for testing
                          }

                          TCP_R.SendBytes(MyBuffer); // send data back to server
                      }
                  }

                  TCP_R.Close(); // tell server that we're done sending data

TCP_R is a class that handles all the TCP stuff (connect to server/send data)

This is the TCP_R.SendBytes function:

    public void SendBytes(Byte[] data)
    {
        try
        {
            if (m_clientSocket != null)
            {
                m_clientSocket.Send(data);
            }

        }
        catch (SocketException se)
        {
            log("send_bytes err: " + se.Message);
        }
    }

Notice I used test_buffer to collect all the bytes to verify if everything is received correctly. If I use this code:

                  MemoryStream ms = new MemoryStream(test_buffer.ToArray());
                  Image returnImage = Image.FromStream(ms);
                  returnImage.Save("image.jpg");

The image is created successfully. So I assume client sends out data correctly.

SERVER Side:

I collect the data using OnDataReceived function and build all the data in REQUEST_RESPONSE which is List<byte>:

  public void OnDataReceived(IAsyncResult asyn)
    {

        try
        {
            SocketPacket socketData = (SocketPacket)asyn.AsyncState; // cast

            int iRx = 0;

                iRx = socketData.m_currentSocket.EndReceive(asyn);
                char[] chars = new char[iRx + 1];

                System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder(); // CAN THIS BE CAUSING THE PROBLEM!?!
                int charLen = d.GetChars(socketData.dataBuffer, 0, iRx, chars, 0);


                for (int s = 0; s < charLen; s++)
                {

                    RESPONSE_DATA.Add(socketData.dataBuffer[s]); // Collect response data

                }

                WaitForData(socketData.m_currentSocket, socketData.socket_id, REQUEST_INDEX_ID);

            }

        }
        catch (ObjectDisposedException)
        {

        }
        catch (SocketException se)
        {

        }
    }

UPDATE2: It appears that these lines are causing the problem:

                System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder(); // CAN THIS BE CAUSING THE PROBLEM!?!
                int charLen = d.GetChars(socketData.dataBuffer, 0, iRx, chars, 0);

How can I fetch the data into socketData.dataBuffer without using Decoder?

SOLUTION: Even though I was reading directly from dataBuffer, the 'd.GetChars(socketData.dataBuffer..' line somehow screwed up the buffer and therefor I got screwed up data. I still needed to use GetChars too to read first part of packet as string so I simply changed:

 int charLen = d.GetChars(socketData.dataBuffer, 0, iRx, chars, 0);

to:

 byte[] tmp_a = new byte[socketData.dataBuffer.Length];
 tmp_a = socketData.dataBuffer;
 int charLen = d.GetChars(tmp_a, 0, iRx, chars, 0);

And now everything works smoothly. Apparently somehow calling getChars directly on buffer affected the buffer.

Answer 1

Sounds like an encoding problem.

The key concept here is that encoding is for text, and images are not text.

Do not try to encode or decode the bytes before sending or after receiving.

Similarly, if you read or write the image from or to a file, do not try to encode or decode, use a binary mode.

Just don't try to convert the bytes to chars, there is no meaningful way to do so.