NSURLCache caching random responses that should not be cached

Question

I am implementing an app which does a lot of networking calls to a rest-api that we also control. Recently we decided to introduce caching headers on the server side to save some valuable networking and server time. As we do not know beforehand for how long the data will be valid, we are not sending Cache-control: max-age or Expires headers, all we do is send a Last-Modified header together with a E-tag, so we always hit the server but responses are pretty fast most of the times with a 304. Everything seemed to work fine at first, with many requests being cached. However, I am experiencing some random data errors on the app due to the caching.

For some reason I can not understand, at some point requests are being locally cached and used as "updated" data without hitting the server, when they actually are not. The problem keeps there until some time passes. Then everything goes to server normally again, exactly as it would behave with a cache-control header, but without it!. So, my question is:

---

How can NSURLCache together with NSURLConnection decide that a particular request does not need to go online when the original request did not come with Cache-control: max-age or Expires headers? Has anyone experienced similar effects? And how can I solve it without removing the whole cache?

---

Some more background info:

• I am using AFNetworking, but it relies on NSURLConnection so I do not think it changes anything
• The cache used is the default [NSURLCache sharedURLCache] instance

• It is a GET request, and when I check the headers from the cached response this is what I get:

  po [response allHeaderFields]

  "Access-Control-Allow-Headers" = "Content-Type";
  "Access-Control-Allow-Methods" = "GET, POST, DELETE, PUT";
  "Access-Control-Allow-Origin" = "*";
  Connection = "keep-alive";
  "Content-Encoding" = gzip;
  "Content-Length" = 522;
  "Content-Type" = "application/json";
  Date = "Mon, 02 Sep 2013 08:00:38 GMT";
  Etag = "\"044ad6e73ccd45b37adbe1b766e6cd50c\"";
  "Last-Modified" = "Sat, 31 Aug 2013 10:36:06 GMT";
  Server = "nginx/1.2.1";
  "Set-Cookie" = "JSESSIONID=893A59B6FEFA51566023C14E3B50EE1E; Path=/rest-api/; HttpOnly";
  

• I can not predict or reproduce when the error is going to happen so solutions that rely on deleting the cache are not an option.

• I am using iOS5+

Answer 1

please make sure that your NSURLRequest cache policy is set to NSURLRequestReturnCacheDataElseLoad

here if you are using AFNetworking in AFHTTPClient.m you can override the method

- (NSMutableURLRequest *)requestWithMethod:(NSString *)method
                                  path:(NSString *)path
                            parameters:(NSDictionary *)parameters

replace the line 470 with this

    NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:url cachePolicy:NSURLRequestReturnCacheDataElseLoad timeoutInterval:15];

what you are actually doing is to telling the request to load the cache if server is not updated..if server is updated then it will ignore the cache and download the content from server

FYI: NSURLCache stores the data in memory..if you want to store the data in disc you can use my class here

https://github.com/shoeb01717/Disc-Cache-iOS

Answer 2

How can NSURLCache together with NSURLConnection decide that a particular request does not need to go online when...

Section 13.2 of RFC 2616 says:

Since origin servers do not always provide explicit expiration times, HTTP caches typically assign heuristic expiration times, employing algorithms that use other header values (such as the Last-Modified time) to estimate a plausible expiration time. The HTTP/1.1 specification does not provide specific algorithms, but does impose worst-case constraints on their results. Since heuristic expiration times might compromise semantic transparency, they ought to used cautiously, and we encourage origin servers to provide explicit expiration times as much as possible.

So, it's possible for the URL loading system to decide that the cached data is "fresh enough" even though you haven't provided a specific lifetime for the data.

For best results, you should try to provide a specific lifetime in your response headers. If adding such a header is impossible, perhaps you could change the request instead. if-modified-since or cache-control could each help you avoid cached data.

Answer 3

According to the statement "at some point requests are being locally cached and used as "updated" data without hitting the server" I am pretty sure that your requests are memory cached.

NSURLCache caches the data in memory. not on disk. So let me explain what might be happening with you.

You launches the app. Makes a web service call it fetches the data from server you again makes a call and it fetches the response from memory without making a call to server and displays you the result.

You leaves the app for sometime or restarts the app. It checks if data is their in memory. If it is not available then it again makes a call to server and repeats the same behaviour.

I would recommend you to write your own disk caching for the same instead of relying on NSURLConnection and NSUrlCache. because some of the caching policies aer still not implemented from Apple.