CODEWITHSUNDEEP
pythongoogle-app-enginepython-2.7httplibhttpexception
Bishan
Bishan

Reputation: 15740

Deadline exceeded while waiting for HTTP response from URL: HTTPException

I'm developing an app to listen for twitter hash tags using tweepy. I have uploaded my app to Google App Engine and it's giving me below error.

Last line of Traceback:

File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/gae_override/httplib.py", line 524, in getresponse
    raise HTTPException(str(e))
HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://stream.twitter.com/1.1/statuses/filter.json?delimited=length

How could I solve this?

Upvotes: 1

Views: 2219

Answers (2)

Zdenek Maxa
Zdenek Maxa

Reputation: 1399

I am running a simple app on GAE which interacts with a jenkins CI server, using jenkinsapi library which depends on requests. I am shipping both jenkinsapi as well as requests with my app, requests is not supported on GAE though it exists in Google Cloud SDK where I took it from. jenkinsapi sends a sick number of requests to the server, I was getting very often 

File "/base/data/home/apps/s~jenkins-watcher/v0-1.382631715892564425/libs/requests-2.3.0-py2.7.egg/requests/adapters.py", line 375, in send
    raise ConnectionError(e, request=request)
    ConnectionError: HTTPConnectionPool(host='XXXXXXX', port=8080):
    Max retries exceeded with url: XXXXXX
    (Caused by <class 'gae_override.httplib.HTTPException'>:
    Deadline exceeded while waiting for HTTP response from URL: XXXXXXXX

It turned out that number of retries is 0 and timeout was some very low default. Increasing both numbers, for which I had to patch the library, helped and I am not seeing this problem anymore.

Actually, it still happens, but:

Retrying (3 attempts remain) after connection broken by 'HTTPException('Deadline exceeded while waiting for HTTP response from URL ...

Upvotes: 0

Bardia D.
Bardia D.

Reputation: 693

You can set the default timeout for url fetch, I believe it's set to 5 seconds by default. That endpoint call might take longer. Perhaps 30 seconds?

urlfetch.fetch(url=url, method=urlfetch.GET, deadline=30)

You can go up to 60 per the docs: https://cloud.google.com/appengine/docs/python/urlfetch/#Python_Fetching_URLs_in_Python

Upvotes: 3

Related Questions