CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-cloudfront
Greg
Greg

Reputation: 33650

Content-Encoding header not returned from Cloudfront

I'm trying to deliver compressed CSS and JS files to my web app. The files are hosted on S3, with a Cloudfront distribution in front of the S3 origin to provide edge cacheing. I'm having trouble getting these files to the browser both compressed and with the right cache-related headers to allow the browser to cache as well.

I have a cloudfront distribution with S3 as the Origin to deliver the JS and CSS files for my web app. I initially set up CloudFront to compress the files, but it would not send the Cache-Control or ETag headers in the response.

Since I also wanted to leverage the browser cache too, I thought of storing the gzipped files in S3, with the Cache-Control, and Content-Encoding headers attached. I did this, and the CloudFront did start returning the Cache-Control and ETag headers in the response, but it would not return the Content-Encoding: gzip header in the response (that I set in the file metadata in S3). Because this header is missing in the response, the browser doesn't know to uncompress the response and ends up with an unreadable file.

I've also tried setting up a viewer response edge lambda to add the Content-Encoding header, but this is disallowed (see the AWS docs) and results in a LambdaValidationError.

Is there something I'm missing here that would allow the files to make it to the browser with compression, AND still allow the Cache-Control and ETag headers to make it through to the browser?

Any help would be much appreciated!

Upvotes: 4

Views: 3925

Answers (2)

Marek Barczyk
Marek Barczyk

Reputation: 11

In your particular case I think you are missing one bit. You need to modify you distrubtion in cloudfront like this: -> Edit the default behavior [*] and select "Cache Based on Selected Request Headers" to Whitelist the "Accept-Encoding" header.

In general the way caching in CloudFront is:

If you have compression enabled in CloudFront, all files which can be compressed, meaning:

will be compressed by CloudFront and will have etag header removed by default. CloudFront does not touch/modify cache-control header which you can set as attribute in your s3 objects.

It might be confusing when diagnosing the disappearance of etag with with curl. Curl by default will return etag because it does not send header:

"Accept-Encoding: gzip, deflate, br"

until you specify it. For non-compressed content etag is preserved by CloudFront.

One thing you can do to have etag is to disable compression on cloudfront but it means increased cost, higher load times.

Other thing is to is to white-list Accept-Encoding header on cloudfront: -> Edit the default behavior [*] and select "Cache Based on Selected Request Headers" to Whitelist the "Accept-Encoding" header and upload compressed s3 object. Remember to setup "Content Encoding" metadata accordingly. Here you will find an instruction: https://medium.com/@graysonhicks/how-to-serve-gzipped-js-and-css-from-aws-s3-211b1e86d1cd

From now on CloudFront will keep cached version and share etag. More reading: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/header-caching.html?shortFooter=true#header-caching-web-compressed

Additionally CloudFront adds:

last-modified: Sat, 13 Jul 2019 07:11:35 GMT header.

If you have cache-control present without etag that the caching behavior works as described here: https://developer.mozilla.org/pl/docs/Web/HTTP/Headers/Cache-Control

If you have only last-modified than it is not 100% obvious how long the browser will cache such request.

Based on my experience when firefox and chrome have this object already cached, when retrieving such object again from CloudFront will add request header:

if-modified-since: Sat, 13 Jul 2019 07:11:35 GMT

CloudFront will respond will proper data if it was modified after this date.

On IE it seems like heuristic caching algorithm is used, you can read more about it here: https://paulcalvano.com/index.php/2018/03/14/http-heuristic-caching-missing-cache-control-and-expires-headers-explained/.

For IE the object can be cached for as long as: (current time - last-modified) * 10%.

Upvotes: 1

Matt
Matt

Reputation: 36

The way I usually do this is to upload uncompressed content to the S3 bucket and put Cache-Control headers on your items there. The Cache-Control header is the only thing I set in the origin (S3).

In Cloudfront I check the 'Compress Objects Automatically' option in Behavior Settings to have Cloudfront compress the files for me. That takes care of the Content-Encoding and Last-Modified headers and the gzipping. That should be all you need. You won't see an ETag header from Cloudfront but Last-Modified does essentially the same thing here.

If you don't see your changes coming through, check that you properly invalidated your Cloudfront cache. I see a lot of people put / in the box but it's really /* to invalidate the entire distribution.

https://aws.amazon.com/about-aws/whats-new/2015/05/amazon-cloudfront-makes-it-easier-to-invalidate-multiple-objects/

This should take care of gzipping, caching from the CDN and browser caching.

Good luck!

Upvotes: 2

Related Questions