Reputation: 6752
So, I'm working with urllib2, and it keeps freezing on a specific page. Not even Ctrl-C will cancel the operation. It's throwing no errors (I'm catching everything), and I can't figure out how to break it. Is there a timeout option for urllib2 that defaults to never?
Here's the procedure:
req = urllib2.Request(url,headers={'User-Agent':'...<chrome's user agent string>...'}) page = urllib2.urlopen(req) // p.s. I'm not installing any openers
Then, if the internet gets cut partway through the second line (which downloads it), even if connection is restored, this freezes the program completely.
Here's the response header I get in my browser (Chrome) from the same page:
HTTP/1.1 200 OK Date: Wed, 15 Feb 2017 18:12:12 GMT Content-Type: application/rss+xml; charset=UTF-8 Content-Length: 247377 Connection: keep-alive ETag: "00e0dd2d7cab7cffeca0b46775e1be7e" X-Robots-Tag: noindex, follow Link: ; rel="https://api.w.org/" Content-Encoding: gzip Vary: Accept-Encoding Cache-Control: max-age=600, private, must-revalidate Expires: Wed, 15 Feb 2017 18:12:07 GMT X-Cacheable: NO:Not Cacheable Accept-Ranges: bytes X-Served-From-Cache: Yes Server: cloudflare-nginx CF-RAY: 331ab9e1443656d5-IAD
p.s. The url is to a large WordPress feed which, according to the response, appears compressed.
Upvotes: 0
Views: 87