Write a basic HTTP Server using socket API

Question

I have this following code which is in my server.py file. It is waiting to receive data from the client. Additionally I cannot use any of the http libraries. Only the socket Library:

def handle_client(conn, addr):
    print ('New client from', addr)
    x = []
    try:
        while True:
            data = conn.recv(1024)
            decoded_data = data.decode('utf-8')
            # if "GET / " in data.decode('utf-8'):
            #   handle_client_get(conn)
            # else:
            if data:
                print(data)
                x.append(decoded_data)
            else:
                print(x)
                break
    finally:
        print("in close now")
        conn.close()

The problem I am having is that I can only reach the print(x) statement once I manually CTRL + Cclose the client. Otherwise it doesn't print.

Why is that the case.

ANSWER

You need to send an acknowledgment to the client such that the data sent has been received correctly.

that will terminate the connection and not wait for a timeout.

This is because the client sends: Expect: 100-continue And you need to send an acknowledgment back to the client

Answer 1

You need to implement the protocol which will have a means to tell you how much data there is to read. In the case of HTTP, the request starts with a CRLF delimited header, and we can read that to get the information we want.

w3.org has a good description of the http request protocol. Its more complicated than I want to implement here but I've included an example that pulls in the request header by reading the socket one character at a time and looking for an empty \n terminated line. By reading one character at a time, I don't have to implement my own line buffer.

The first line is the request method and the remaining lines are other parameters included with the request. For a POST, for instance, there would be more data still to read from the socket.

import re

def handle_client(conn, addr):
    print ('New client from', addr)
    header = []
    line = []
    try:
        # read ascii http client header. 
        while True:
            c = conn.recv(1)
            # check for early termination
            if not c:
                return None
            # check for end of request line
            elif c == b"\n":
                # make line a string to add to header
                line = ''.join(line).decode('ascii').strip()
                # are we at the empty line signalling end-of-header?
                if not line:
                    break
                header.append(line)
                line = []
            # filter out \r
            elif c == b"\r":
                continue

        # request is first line of header
        request_line = header.pop(0)
        method, uri, http_version = re.split(r" +" request_line)

        if method.upper() == "GET":
            # this function needs more parameters... the uri to get and the protocol
            # version to use.
            handle_client_get(...)
    finally:
        print("in close now")
        conn.close()

Answer 2

Edit:
Now that you mention this is a HTTP server and is to be done using only socket library, I suggest you remove certain complex elements from the program at this stage like removing the while block until this problem is solved.

server.py

def handle_client(conn, addr):
    print ('New client from', addr)
    x = []
    try:
        data = conn.recv(1024)
        decoded_data = data.decode('utf-8')
        if data:
            print(data)
            x.append(decoded_data)
        else:
            print(x)
            break
    finally:
        print("in close now")
        conn.close()

Original Answer:

Try sending an additional \n at the sender end and use the flush function later to send out any remaining data on the buffer something like:

sender.py

def send_data(data,sock_obj,addr):
    sock_obj.send(data+"\n")
    sock_obj.flush()