CODEWITHSUNDEEP
c#amazon-web-servicesaws-lambdaamazon-cloudwatchnlog
Luke
Luke

Reputation: 23680

How do I configure an AWS Lambda in a VPC to be able to log to another Cloudwatch log group?

How do you configure an AWS Lambda that is in a VPC to be able to log to a Cloudwatch log group other than the one that is associated with the lambda itself?

This is required so that I can avoid the cost of the summary logs that are created with every request, eg:

2022-02-10T10:16:19.133+00:00   START RequestId: 52cbfd7e-71ec-4499-a292-7c81c94535f3 Version: $LATEST
2022-02-10T10:16:24.250+00:00   END RequestId: 52cbfd7e-71ec-4499-a292-7c81c94535f3
2022-02-10T10:16:24.250+00:00   REPORT RequestId: 52cbfd7e-71ec-4499-a292-7c81c94535f3 Duration: 5100.10 ms Billed Duration: 7412 ms Memory Size: 256 MB Max Memory Used: 201 MB Init Duration: 2311.15 ms

So I wish to log my own logs to a separate Cloudwatch log group and add a Deny policy to prevent logging to the default Cloudwatch group that's associated with the lambda by default.

I'm using NLog, so I've done the following so far (following Nlog instructions from here):

The contents of nlog.config:

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      throwConfigExceptions="true">
  <extensions>
    <add assembly="AWS.Logger.NLog" />
  </extensions>
  <targets>
    <target name="logconsole" xsi:type="Console" />
    <target name="aws" type="AWSTarget" logGroup="my-lambda-logs" region="us-east-1"/>
  </targets>
  <rules>
    <logger name="*" minlevel="Info" writeTo="logconsole,aws" />
  </rules>
</nlog>

When I run my lambda at the moment I'm getting a timeout because it is not able to log to Cloudwatch from within a VPC.

Upvotes: 1

Views: 1697

Answers (2)

diklwiek3983
diklwiek3983

Reputation: 11

When a Lambda function is configured to run inside a VPC, the function can access resources inside the VPC such as databases. However, it does not have direct internet access by default.

To allow the Lambda function to send logs to CloudWatch Logs, AWS sets up a private connection between the VPC and CloudWatch Logs using interface VPC endpoints. This allows the logs to be sent to CloudWatch Logs privately without going over the public internet.

The user does not need to manually create a VPC endpoint for CloudWatch Logs. AWS creates and manages this endpoint automatically when Lambda functions are configured to run inside a VPC.

Upvotes: 1

Luke
Luke

Reputation: 23680

  1. If you haven't already created your lambda, create it within your VPC, specify your private subnet ids and attach a security group that allows outbound (egress) communication on at least port 443 TCP aka HTTPS.

  2. Within VPC on the AWS console, create a VPC Endpoint for Cloudwatch Logs in the region that you wish to log to in the same subnets as your lambda VPC configuration. This will allow your lambda to use the normal cloudwatch endpoint from within your VPC without any log traffic leaving the AWS network.

AWS VPC endpoint creation for cloudwatch logs

  1. Ensure that you set Private Dns Enabled to true (you can do this by modifying after creation if you can't find this during creation)

Enable private DNS names for VPC endpoint

  1. Specify a security group for your new VPC endpoint that includes an inbound (ingress) rule to allow the security group of your lambda to communicate over port 443.

VPC Endpoint security group rule inbound

  1. Now that you can communicate with Cloudwatch, you just need to create your log group and give your lamda a policy that allows it to log to your cloudwatch log group.

    {
 "Version": "2012-10-17",
 "Statement": [
 {
   "Action": [
     "logs:CreateLogStream",
     "logs:PutLogEvents",
     "logs:DescribeLogGroups"
   ],
   "Resource": [
      "arn:aws:logs:us-east-1:369074612400:log-group:my-lambda-logs:*", 
      "arn:aws:logs:us-east-1:369074612400:log-group:my-lambda-logs:*:*"
   ],
   "Effect": "Allow"
 }
]

    }

For the sake of this example the ARN of the cloudwatch log group is:

arn:aws:logs:us-east-1:000000000001:log-group:my-lambda-logs

... but you can get yours from the AWS console (it will contain your account id and the correct region of the Cloudwatch group.

Voilá your lambda should now be able to use NLog (or another logging framework) to log to Cloudwatch logs.

Upvotes: 2

Related Questions