CODEWITHSUNDEEP
pythonhttpstreamingclientkeep-alive
Matteo Caprari
Matteo Caprari

Reputation: 2969

Python stream http client with keep-alive

I need a python http client that can reuse connections and that supports consuming the stream as it comes in. It will be used to parse xml streams, sax style.

I came up with a solution, but I'm not sure it is the best one (there are quite a few ways of writing an http client in python)

class Downloader():

    def __init__(self, host):
            self.conn = httplib.HTTPConnection(host)

    def get(self, url):
            self.conn.request("GET", url)
            resp = self.conn.getresponse()
            while True:
                    data = resp.read(10)
                    if not data:
                            break
                    yield data

Thanks folks!

Upvotes: 1

Views: 5652

Answers (2)

Ignacio Vazquez-Abrams
Ignacio Vazquez-Abrams

Reputation: 798686

urlgrabber supports keepalive and can return a file-like object.

Upvotes: 1

Bryce
Bryce

Reputation: 320

There is also pycurl. By default keepalive is turned on and you can write to a file for output.

Follow the examples, they are quite helpful

Upvotes: 1

Related Questions