CODEWITHSUNDEEP
pythonpython-requests
mgilbert
mgilbert

Reputation: 3645

Diagnosing hanging requests with requests

I have seen several related posts on dealing with hanging request using the timeout argument, however I am unsure how to diagnose whether my request is hanging or if there is some other underlying problem. I am streaming some data and logging it as well as printing it to console. Twice now I have seen extended periods where no data is printed to console and when I interupt execution I get the following stacktrace

  File "/home/matthew/Dropbox/pylibs/oandapy/oandapy.py", line 276, in start
    for line in response.iter_lines(90):
  File "/home/matthew/anaconda/lib/python2.7/site-packages/requests/models.py", line 663, in iter_lines
    for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
  File "/home/matthew/anaconda/lib/python2.7/site-packages/requests/models.py", line 627, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/home/matthew/anaconda/lib/python2.7/site-packages/requests/packages/urllib3/response.py", line 240, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/home/matthew/anaconda/lib/python2.7/site-packages/requests/packages/urllib3/response.py", line 187, in read
    data = self._fp.read(amt)
  File "/home/matthew/anaconda/lib/python2.7/httplib.py", line 543, in read
    return self._read_chunked(amt)
  File "/home/matthew/anaconda/lib/python2.7/httplib.py", line 585, in _read_chunked
    line = self.fp.readline(_MAXLINE + 1)
  File "/home/matthew/anaconda/lib/python2.7/socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
  File "/home/matthew/anaconda/lib/python2.7/ssl.py", line 246, in recv
    return self.read(buflen)
  File "/home/matthew/anaconda/lib/python2.7/ssl.py", line 165, in read
    return self._sslobj.read(len)

Is there some additional parameters I can set or somewhere to log more information relating to the connection, it is not clear to me from the stacktrace what the problem is.

Upvotes: 1

Views: 1374

Answers (1)

pmod
pmod

Reputation: 10997

You can set timeout for HTTP session when you create Request object or send the request; see here:

timeout – (optional) Float describing the timeout of the request in seconds.

However, it looks like something wrong during transmission itself. For testing purposes, you can pass chunk_size=1 to iter_lines, and immediately print the result. It's possible that server doesn't send HTTP chunked terminator \r\n (check if you have own server? worth trying other servers through https).

Upvotes: 1

Related Questions