CODEWITHSUNDEEP
pythontcptwistedhttpserver
Ivan Kalchev
Ivan Kalchev

Reputation: 395

HTTP Server with active TCP connections in python

I am writing a pseudo-http application in python, the requirements for which are:

  1. It should handle HTTP requests.
  2. The connections between the client and the server outlive the request-response, i.e. the underlying TCP connection remains alive after a response has been sent to a client.
  3. The server needs to be able to send data to a particular client for which it already has an opened connection.

I looked at twisted and python's TCPServer/BaseHTTPServer, but they don't quite fit the bill. The way I see it, I have two options:

  1. Start from a HTTP server implementation and override my way down to connection management.
  2. Have a simple socket server that will manage the connections and pass data between the "http" server and the client.

Has anyone tackled a similar issue? Any ideas on other approaches or which one will be a better option?

Thanks!

EDIT 1 I cannot use HTTP 2 or web sockets; HTTP <2 over TCP is a hard requirement.

Upvotes: 1

Views: 3615

Answers (2)

Ivan Kalchev
Ivan Kalchev

Reputation: 395

I ended up overriding methods in http.server.HTTPServer, it was less work than expected and it's all from the standard packages.

Depending on your situation the below could end up being more involved, e.g. using more structured session representation, etc. Then again you should probably consider more developed frameworks like twisted in that case.

The main points are:

  • Use a ThreadingMixIn - as the connections are long-lived, a separate handler thread would be needed in order to take more than one connection at a time.
  • Note that if you are using the BaseHTTPRequestHandler, the connection is closed after each response, unless there is a Connection: keep-alive header or you set self.close_connection = False on EVERY request.

Anyway, a snippet to get you started:

from http.server import HTTPServer, BaseHTTPRequestHandler
from socketserver import ThreadingMixIn

class MyHandler(BaseHTTPRequestHandler):

   # Implement do_GET, do_POST, etc.

   def handle_one_request(self):
      super(MyHandler, self).handle_one_request()
      self.close_connection = some_condition()
      if self.close_connection:
         # Remove the session from the server as it will be closed after this
         # method returns
         self.server.sessions.pop(self.client_address)

class MyServer(ThreadingMixIn, HTTPServer):
   def __init__(self, addr_port, handler_class):
      super(MyServer, self).__init__(addr_port, handler_class)
      self.sessions = {} # e.g. (addr, port) -> client socket

   def get_request(self):
      """Just call the super's method and cache the client socket"""
      client_socket, client_addr = super(MyServer, self).get_request()
      self.sessions[client_addr] = client_socket
      return (client_socket, client_addr)

   # You may also want to add the following
   def server_close(self):
      """Close any leftover connections."""
      super(MyServer, self).server_close()
      for _, sock in self.sessions.items():
         try:
            sock.shutdown(socket.SHUT_WR)
         except socket.error:
            pass
         sock.close()

Upvotes: 0

L. Daniel Burr
L. Daniel Burr

Reputation: 11

Since you cannot use websockets or http/2, and you require the ability to push data from the server to the client, then long-polling is probably the best option remaining.

See Nevow, at https://github.com/twisted/nevow, for one possible implementation of long-polling, via the athena module.

Upvotes: 1

Related Questions