CODEWITHSUNDEEP
pythoncachinghttp-headersdebian
Evan Marks
Evan Marks

Reputation: 221

How to prevent python urllib3 from caching response

I am polling the same url several times in a row with urllib3. I have profiled the requests and I am getting the header:

Cache-Control: no-transform, max-age=120

On requests after the first I am getting the cached version of the page rather than the request being run again. I have no control over the headers returned from the server, how can I prevent the caching?

I am on Debian Squeeze and this is running python 2.6.6

pool = urllib3.HTTPConnectionPool('itunes.apple.com')
request = pool.request('GET', '/webObjects/MZStore.woa/wa/viewTop?
                       selected_tab_index=0&startIndex=0&genreId=36', 
                       headers = {'Host': 'itunes.apple.com', 
                       'Accept-Encoding': 'gzip, deflate', 'X-Apple-Tz': -18000, 
                       'X-Apple-Store-Front': '143441-1,2')

Upvotes: 3

Views: 4414

Answers (2)

James Thiele
James Thiele

Reputation: 413

Try adding the following headers:

Cache-Control: no-cache

Pragma: no-cache

Expires: Thu, 01 Jan 1970 00:00:00 GMT

Upvotes: 2

shazow
shazow

Reputation: 18187

urllib3 has no native caching built in. Are you sure it's not the server responding with cached results?

Upvotes: 3

Related Questions