CODEWITHSUNDEEP
socketspython-2.7client
JDM
JDM

Reputation: 1804

Server response on socket being truncated at 2K

I wrote the following function for use in a collection of socket utilities I use for testing. (Using Python 2.7.3, if that's important.) The main reason I pulled the select library into it was so I could implement a timeout instead of just waiting forever for a response. The problem I've found though is that the response is getting truncated at 2048 characters, despite using 64K as the maximum size for the .recv() method on the socket. This truncation didn't happen before I involved select. It happily pulled through 64K and even more when I set the maximum size higher.

I've looked through some online resources on select and I find any information on this apparent cap on the size of the received data. I.e. no information that it exists, let alone how to modify it. Can anyone point me to a way to overcome this 2K limit?

    import socket,select
    MAXSIZE = 65535
    TIMEOUT = 10

    def transientConnect(host,port,sendData):
        error,response = False,''
        try:
            sendSocket = socket.socket()
            sendSocket.connect((host,port))
            sendSocket.send(sendData)
            gotData = select.select([sendSocket],[],[],TIMEOUT)
            if (gotData[0]):
                response = sendSocket.recv(MAXSIZE)
            else:
                error    = True
                response = '*** TIMEOUT ***\nNo response from host.'
            sendSocket.close()
        except Exception, errText:
            error,response = True,'*** SOCKET ERROR ***\n'+str(errText)
        return (error,response)

Upvotes: 2

Views: 1671

Answers (2)

JDM
JDM

Reputation: 1804

The corrected, tested, and working function:


    MAXSIZE = 65535
    TIMEOUT = 10
    def transientConnect(host,port,sendData):
        error,response = False,''
        try:
            sendSocket = socket.socket()
            sendSocket.connect((host,port))
            sendSocket.send(sendData)
            gotData = select.select([sendSocket],[],[],TIMEOUT)
            if (gotData[0]):
                response = sendSocket.recv(MAXSIZE)
                while True:
                    #Once data starts arriving, use a shorter timeout
                    gotData2 = select.select([sendSocket],[],[],0.5)
                    if (gotData2[0]):
                        moreData = sendSocket.recv(MAXSIZE)
                        response += moreData
                    else:break
            else:
                error    = True
                response = '*** TIMEOUT ***\nNo response from host.'
            sendSocket.close()
        except Exception, errText:
            error,response = True,'*** SOCKET ERROR ***\n'+str(errText)
        return (error,response)

Upvotes: 1

Nikolai Fetissov
Nikolai Fetissov

Reputation: 84189

TCP socket gives you a bi-directional stream of bytes, but at the lower level it is being realized over packet-based protocol (IP). That means you can receive arbitrary chunks of the stream at each read (the bytes are all in-order though) depending on how the TCP/IP stack decided to split the stream for transmission.

The critical point here is that you have to do recv() in a loop until you get a complete "application message". To quote from Socket Programming HOWTO:

Now we come to the major stumbling block of sockets - send and recv operate on the network buffers. They do not necessarily handle all the bytes you hand them (or expect from them), because their major focus is handling the network buffers. In general, they return when the associated network buffers have been filled (send) or emptied (recv). They then tell you how many bytes they handled. It is your responsibility to call them again until your message has been completely dealt with.

What you are seeing here is that select gives you early notification - as soon as some data is available.

Hope this helps.

Upvotes: 3

Related Questions