CODEWITHSUNDEEP
httpamazon-cloudfrontuser-agent
Tom Raganowicz
Tom Raganowicz

Reputation: 2470

AWS CloudFront - forward User-Agent but don't cache against it

I want my origin to be able to see the User-Agent header .e.g: Gecko/20100101 Firefox/62.0 not Amazon CloudFront.

In the Behaviors tab I can whitelist User-Agent header, so it's passed to the origin correctly, however now CloudFront caches content per User-Agent, meaning that user visiting the CloudFront endpoint from different browsers forces CloudFront to go to the origin.

Is there any way to configure CloudFront to pass some headers to the origin, but not necessarily cache against them?

EDIT: I've got similar problem with Accept-Language header. I want to pass it to the origin, however I don't want to cache against it. Assets that I am caching are not Language dependent, however the non-cachable content is dependent on the Accept-Language header.

Upvotes: 23

Views: 20961

Answers (4)

domis86
domis86

Reputation: 1357

It seems that around July 2020 Amazon introduced new feature "Origin Request policies". You should be able to use it (without bogging down in Lambda@Edge): https://aws.amazon.com/blogs/networking-and-content-delivery/amazon-cloudfront-announces-cache-and-origin-request-policies/

TLDR:

Over time, we’ve seen numerous cases in which the new functionality could be useful for customers. Examples such as:

  • Forwarding information such as the User-Agent to the origin for analytics/logging but without serving different content variants based on device type (now you can forward the user-agent header and exclude it from the cache-key)

^ this is your use-case :)

Upvotes: 17

illagrenan
illagrenan

Reputation: 6555

You can use Lambda@Edge function (https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html) assigned to your CloudFront distribution. You would need two functions:

  1. Viewer-Request event handler, that will read User-Agent header and copy it to e.g. X-My-User-Agent. Viewer-Request handler is invoked before the request from the client reaches your Cloudfront Distribution.
  2. Origin-Request event handler, that will read X-My-User-Agent and replace User-Agent. Origin-Request handler is invoked when Cloudfront did not find requested page in its cache and sends the request to the origin.

Please note that you should NOT add User-Agent to Cloudfront whitelist:

You can configure CloudFront to cache objects based on values in the Date and User-Agent headers, but we don't recommend it. These headers have a lot of possible values, and caching based on their values would cause CloudFront to forward significantly more requests to your origin.

Ref: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/RequestAndResponseBehaviorCustomOrigin.html#request-custom-headers-behavior

Example of Viewer-Request handler (Lambda@Edge can be written only in NodeJS or Python, Ref: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html#lambda-requirements-lambda-function-configuration):

'use strict';

exports.handler = (event, context, callback) => {
  const request = event.Records[0].cf.request;
  const headers = request.headers;
  const customUserAgentHeaderName = 'X-My-User-Agent';
  const userAgent = headers['user-agent'][0].value;

  headers[customUserAgentHeaderName.toLowerCase()] = [
    {
      key: customUserAgentHeaderName,
      value: userAgent
    }
  ];


  callback(null, request);
};

Example of Origin-Request handler:

'use strict';

exports.handler = (event, context, callback) => {
  const request = event.Records[0].cf.request;
  const headers = request.headers;
  const customUserAgentHeaderName = 'X-My-User-Agent';
  const realUserAgent = headers[customUserAgentHeaderName.toLowerCase()][0].value;

  headers['user-agent'] = [
    {
      key: 'User-Agent',
      value: realUserAgent
    }
  ];


  callback(null, request);
};

Upvotes: 26

Ibrahimsha
Ibrahimsha

Reputation: 310

This might be a simple solution. If you wanted User-Agent for a unique type of URLs example, /tracking/a,/tracking/b like this create a new distribution for this path [tracking*] and whitelist User-Agent for this distribution only. So you are not messing with AWS caching for all urls but for this path only.

Upvotes: 0

cnvzmxcvmcx
cnvzmxcvmcx

Reputation: 1111

If the requests are cached across different user-agents, in case of a hit, the real-user agent will not be passed to the origin at all. CloudFront will just return the cached response.

You mentioned that you like to send the user-agent information to Elasticsearch. Unless you are only interested in the requests that are missed, you can not rely on the logs collected from the origin application.

If you have Lambda@Edge to send user-agent as realUserAgent, but the user-agent header is itself not a caching parameter, the origin will still not receive that data in case of a Miss.

The only solution that I see here, is to use the access logs generated from CloudFront. The CloudFront access logs contain not only user-agent but also IP addresses and other useful information. This data is logged for both Hit and Miss. It is also easy to set up a logstash to send this information to Elasticsearch.

Upvotes: 3

Related Questions