Browser caching request/response without any Cache-Control header

Question

I am experiencing a browser cache issue with a SPA web app and REST APIs backend. I can reproduce it on Firefox and Safari with the developer tools opened: I made sure that cache is NOT disabled.

When I go for the first to a specific page that is simply fetching and display an object from the REST APIs, I do it with "hard refresh" (CMD+R on Mac). I see the following headers:

1st request:

Host: localhost:5000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko/20100101 Firefox/77.0
Accept: application/json, text/plain, */*
Accept-Language: en,it;q=0.7,fr;q=0.3
Accept-Encoding: gzip, deflate, br
Origin: http://localhost:3000
DNT: 1
Connection: keep-alive
Referer: http://localhost:3000/literature/sde5e-zeb98
Cookie: ...
If-Modified-Since: Fri, 10 Jul 2020 16:19:24 GMT
If-None-Match: "2"
Cache-Control: max-age=0

(notice the Cache-Control header, added automatically because of the hard refresh)

Response:

HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 6128
ETag: "2"
Last-Modified: Fri, 10 Jul 2020 16:19:24 GMT
Link: <https://localhost:5000/api/documents/sde5e-zeb98>; rel="self"
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 52
X-RateLimit-Reset: 1594398279
Retry-After: 60
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31556926; includeSubDomains
Referrer-Policy: strict-origin-when-cross-origin
Access-Control-Allow-Origin: http://localhost:3000
Access-Control-Expose-Headers: Content-Type, ETag, Link, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
Access-Control-Allow-Credentials: true
Vary: Origin
Server: Werkzeug/1.0.1 Python/3.6.7
Date: Fri, 10 Jul 2020 16:23:38 GMT

The request reaches the backend as expected, I am logging it. Backend returns ETag + Last modified headers, but no other cache headers (for example max-age). I want the browser to always hit the backend.

The issue happens when I do not hit hard refresh, but simply navigate on the website to another page and then navigate again to this page (not via back button), basically a normal usage of the website. I would expect that the browser performs the request to the backend every single time, backend returns the ETag and 200/304 status code and browser uses client cache accordingly (ETag calculation on backend is correct). The problem is that the request does not even reach the backend, there is no request performed, no backend hit and I don't understand why. I can see in Firefox the value cached in the Transferred column.

Her the request:

Host: localhost:5000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko/20100101 Firefox/77.0
Accept: application/json, text/plain, */*
Accept-Language: en,it;q=0.7,fr;q=0.3
Accept-Encoding: gzip, deflate, br
Origin: http://localhost:3000
DNT: 1
Connection: keep-alive
Referer: http://localhost:3000/literature/sde5e-zeb98
Cookie: ...

(no Cache-Control header, that's correct, it should be the normal behaviour)

Cached response (no backend hit):

HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 6128
ETag: "2"
Last-Modified: Fri, 10 Jul 2020 16:19:24 GMT
Link: <https://localhost:5000/api/documents/sde5e-zeb98>; rel="self"
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 52
X-RateLimit-Reset: 1594398445
Retry-After: 60
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31556926; includeSubDomains
Referrer-Policy: strict-origin-when-cross-origin
Access-Control-Allow-Origin: http://localhost:3000
Access-Control-Expose-Headers: Content-Type, ETag, Link, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
Access-Control-Allow-Credentials: true
Vary: Origin
Server: Werkzeug/1.0.1 Python/3.6.7
Date: Fri, 10 Jul 2020 16:26:24 GMT

I am trying to understand which header is stopping the browser to perform the request. I have no Cache-Control header set. I suspect it might be an issue with the Vary header, but any hint is appreciated.

Any idea how to debug why the browser is caching the request? What header is wrong? Thanks!

Answer 1

The lack of Cache-Control or Expires doesn't mean that caching is disabled; it means that the browser is free to implement its own caching policy. As described in RFC 7234:

Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified, employing algorithms that use other header field values (such as the Last-Modified time) to estimate a plausible expiration time.

The caching policy you actually want is Cache-Control: no-cache, so return that.

The response may be stored by any cache, even if the response is normally non-cacheable. However, the stored response MUST always go through validation with the origin server first before using it.