CODEWITHSUNDEEP
pythonunit-testingsimplehttpserver
Einar
Einar

Reputation: 4933

Using SimpleHTTPServer for unit testing

I'm writing a Python module that wraps out a certain web service API. It's all REST, so relatively straightforward to implement.

However, I found a problem when it comes to unit testing: as I don't run the services I made this module for, I don't want to hammer them, but at the same time, I need to retrieve data to run my tests. I looked at SimpleHTTPServer, which would be OK.

I solved part of the issue I had, but now, since I can't seem to terminate the thread, I get issues with "address already in use" when launching the test application more than once.

Here's some sample code

PORT = 8001

handler = SimpleHTTPServer.SimpleHTTPRequestHandler

httpd = SocketServer.TCPServer(("", PORT), handler)

httpd_thread = threading.Thread(target=httpd.serve_forever)
httpd_thread.daemon = True
httpd_thread.start()

api_data = urllib.urlopen("http://localhost:8001/post/index.json")
print "Data start:"
print json.load(api_data)

Where "index.json" is a mock JSON file I made which substitutes the real thing. How can I clean things gracefully after the program terminates?

Upvotes: 18

Views: 6502

Answers (3)

Peter Hansen
Peter Hansen

Reputation: 22107

Try using a subclass of TCPServer with allow_reuse_address set True:

import socketserver

class TestServer(socketserver.TCPServer):
    allow_reuse_address = True

...
httpd = TestServer(("", PORT), handler)

Upvotes: 12

Nick Holden
Nick Holden

Reputation: 3799

Old thread but the answers here didn't help me, I'm using HTTPServer, and shutting down after each unit test (by default HTTPServer has allow_reuse_address = 1 set). However I still got the address already in use problem after calling shutdown. I fixed using:

from BaseHTTPServer import HTTPServer

class MyHTTPServer(HTTPServer):
    def shutdown(self):
        self.socket.close()
        HTTPServer.shutdown(self)

Not sure why this doesn't happen by default? May be this isn't optimal?

Upvotes: 5

S.Lott
S.Lott

Reputation: 392010

We use a server built on wsgiref. http://docs.python.org/library/wsgiref.html

It's very easy to add features to this server as we add unit tests.

We start the server with subprocess. http://docs.python.org/library/subprocess.html?highlight=subprocess#module-subprocess

We do not use threads for this kind of testing. Why? (1) Our unit test server is rather complex and we'd like to keep it completely isolated from client applications. (2) Our client applications will be separate processes on separate hardware, we need to be sure that we have realistic performance expectations for that configuration. (3) It's simpler. (4) It's portable across all platforms. (5) It's trivial to change from stand-alone unit testing to integration testing with a production-like server that's already running.

We actually have a small WSGI application that makes the server shutdown in a reasonably controlled manner so that the logs are shutdown properly.

Upvotes: 4

Related Questions