Python sockets: Server waits for nothing when asked to 'recv' and then 'sendall'

Question

I am experimenting with python sockets to try to understand the whole concept better, but I have run into a problem. I have a simple server and a client, where the client sends a list to the server, and then waits for the server to send a string signaling the process is complete.

This is the client file:

import socket
import json


host = '192.168.1.102'
port = 14314


def request():

    print 'Connecting'
    clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    clientsocket.connect((host, port))

    print 'Sending request'
    clientsocket.sendall(json.dumps([1, 2, 3, 4, 5, 6, 7, 8, 9]))

    print 'Receiving data'
    data = clientsocket.recv(512)
    print 'Received: {}'.format(data)


request()

and here is the server file:

import socket
import json


host = '192.168.1.102'
port = 14314


def run():

    print 'Binding socket'
    serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    serversocket.bind((host, port))

    print 'Waiting for client'
    serversocket.listen(1)
    clientsocket, addr = serversocket.accept()

    print 'Receiving data'
    raw_data = ''
    while True:
        tmp = clientsocket.recv(1024)
        if not tmp:
            break
        raw_data += tmp
    data = json.loads(raw_data)
    print 'Received: {}'.format(data)

    print 'Sending data'
    clientsocket.sendall('done')


run()

The problem is that while the client is done sending the list, the server is stuck in the recv loop, waiting for nothing. The whole data has been received in the first iteration and in the second iteration there is nothing to be received because the client has moved on to the receiving part.

The weird part is that if I comment out the receive part from the client and the send part from the server, the process completes successfully. So, what am I doing wrong? Why is this not working?

Thanks.

Answer 1

The Docs for socket.recv talk about additional flags being able to be passed in to the recv function described in the unix documentation. So turning to that documentation, I found the following message:

If no messages are available at the socket, the receive calls wait for a message to arrive, unless the socket is nonblocking (see fcntl(2)), in which case the value -1 is returned

So once again, we're directed to another page. The documentation for fcntl says

Performs one of the operations described below on the open file descriptor

So, normally the socket.recv function is blocking (it will wait indefinitely for new data), unless we use a file descriptor. How do we do that? Well there is a socket.makefile function that gives us a file descriptor attached to the socket. Cool. This SO question gives us an example of how we can read and write to a socket, using a file descriptor.

Well what if we don't want to use a file descriptor. Reading further into the unix documentation for the recv function, I see that I can use the MSG_DONTWAIT flag. This doesn't work in Windows, but I did find out that we can use socket.setbocking(False) to permamently change the socket to non-blocking mode. You would then need to ignore any "A non-blocking socket operation could not be completed immediately" errors. Those are normal and non-fatal(error #10035 of this page mentions it is non-fatal).

Another possible implementation would be to multi-thread your program, you can implement a receiving and a sending thread for your socket. This might give you the best performance, but it would be a lot of work to setup.

Python is awesome. I just found some libraries Python has that does asynchronous sockets too. There's asyncore, asynchat which have both been deprecated in favor of asyncio if that is available in the version of Python you are using.

Sorry for throwing so much out there. I don't know a whole lot about sockets. I used them once with the Paramiko library, and that was it. But it looks like there are a lot of ways of implementing them.