CODEWITHSUNDEEP
amazon-cloudfrontaws-lambda-edge
Matthew Amato
Matthew Amato

Reputation: 2022

Modify a CloudFront request before logging?

I'm building an ELK stack (for the first time) to track end-user REST API usage for a CloudFront distribution (in front of an S3 origin). Users pass a refresh token as part of their request and I was hoping to use this token to identify which users were making which request. Unfortunately, it looks like CloudFront access logs are missing some header information (particularly Authorization/Accept in my use case). This leaves me with three questions:

  1. Is there a way to tell CloudFront to log additional items? It appears the answer is no.

  2. As an alternative strategy, I tried modifying the request object with lambda@edge (in Viewer Request) to move the header information into the query string (so that it would get logged) but any manipulation in lambda@edge does not seem to be reflected in the log. (though it is reflected in the Origin Request function). Should this be possible?

If doing what I want is impossible, I think the alternative approach is forgo CloudFront logs completely and just fire an http request to logstash with every user request, but I feel like this could be easy to overload.

Thanks

Upvotes: 1

Views: 905

Answers (1)

Matthew Amato
Matthew Amato

Reputation: 2022

After a few days of research and reaching out to Amazon, I was finally able to answer my own questions:

  1. CloudFront logs can't be customized, they are what they are.
  2. See 1.

It turns out that customization is the wrong approach. What I really need to do is aggregate two separate logs that have the information I need into a single logstash entry. It turns out that the Viewer Response lambda@edge function contains a requestId property (actually event.Records[0].cf.config.requestId) which matches the CloudFront log x-edge-request-id column. So while I haven't finished implementing it yet, these two columns can be used in the logstash config for aggregation. I just need to make sure I set up a Viewer Response event that logs out a consistent format that I can then part with logstash. I'm using the logstash-input-cloudwatch_logs to retrieve teh cloudwatch logs.

Upvotes: 3

Related Questions