TCP Socket receiving only a part of the message

Question

I am using the following code to perform a tcp socket connection and send a string to an IP. But sometimes in the response, I not receiving the entire file

     Socket m_socClient;
        IPSelected ="1.1.2.3"
                Port = "80"
                string query ="My Query"
                m_socClient = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                System.Net.IPAddress remoteIPAddress = System.Net.IPAddress.Parse(IPSelected);
                System.Net.IPEndPoint remoteEndPoint = new System.Net.IPEndPoint(remoteIPAddress, Port);
                m_socClient.Connect(remoteEndPoint);
                try
                {
                    if (m_socClient.Connected)
                    {                       
                        var reQuestToSend = "";


                            reQuestToSend = string.Format("POST /TMHP/Request HTTP/1.1\r\nHost:{0}\r\nContent-Length:{1}\r\n\r\n{2}", "edi-webtest.tmhp.com", Query270.Length, Query270);
                        byte[] bytesToSend = Encoding.ASCII.GetBytes(reQuestToSend);
                        byteCount = m_socClient.Send(bytesToSend, SocketFlags.None);
                        byte[] bytesReceived = new byte[3000];
  byteCount = m_socClient.Receive(bytesReceived, SocketFlags.None);
                        Response271 = Encoding.ASCII.GetString(bytesReceived);

                    }
                }
                catch (Exception ex)
                {
                    EVCommon.Log(ex.Message);

                }
                finally
                {
                    m_socClient.Disconnect(false);
                    m_socClient.Close(5000);
                }

I think the problem is with byte[] bytesReceived = new byte[3000]; Is there a way to not hardcode this number 3000. It works most of the time but for longer strings it gets only half of it. I want it be handling variable sized messages instead of setting the byte size to 30000 Thank you

Answer 1

Read RFC 2616 Section 4.4. It tells you how to determine the end of the server's response so you know how many bytes to read. You have to read and process the server's response headers first in order to know how the remaining data, if any, is being transmitted. Then you can keep reading from the socket accordingly, potentially parsing what you have read, until the end of the response has actually been reached. Your current reading code is not even close to satisfying that requirement.

For example (pseudo code):

line = read a CRLF-delimited line;

responseNum = extract from line;
httpVer = extract from line;

do
{
    line = read a CRLF-delimited line;
    if (line == "") break;
    add line to headers list;
}
while (true);

if (((responseNum / 100) != 1) &&
    (responseNum != 204) &&
    (responseNum != 304) &&
    (request was not "HEAD"))
{
    if ((headers has "Transfer-Encoding") &&
        (headers["Transfer-Encoding"] != "identity"))
    {
        do
        {
            line = read a CRLF-delimited line;
            chunkLen = extract from line, decode hex value;
            if (chunkLen == 0) break;
            read exactly chunkLen number of bytes;
            read and discard a CRLF-delimited line;
        }
        while (true);

        do
        {
            line = read a CRLF-delimited line;
            if (line == "") break;
            add line to headers list, overwrite if exists;
        }
        while (true);

        decode/transform read data based on headers["Transfer-Encoding"] values if more than just "chunked"
    }
    else if (headers has "Content-Length")
    {
        read exactly headers["Content-Length"] number of bytes
    }
    else if (headers["Content-Type"] == multipart/byteranges)
    {
        boundary = headers["Content-Type"]["boundary"];
        read and parse MIME encoded data until "--"+boundary+"--" line is reached;
    }
    else
    {
        read until disconnected;
    }
}

if (((httpVer >= 1.1) && (headers["Connection"] == "close)) ||
    ((httpVer < 1.1) && (headers["Connection"] != "keep-alive")))
{
    disconnect;
}

I leave it as an exercise for you to actually implement this in your code.