CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaterraformterraform-provider-aws
Seeker90
Seeker90

Reputation: 895

Write AWS Lambda Logs to CloudWatch Log Group with Terraform

I am trying write the logs of a lambda function into a CloudWatch Log Group created by terraform.

This is the lambda policy json -

{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "Stmt1580216411252",
        "Action": [
          "logs:CreateLogStream",
          "logs:CreateLogDelivery",
          "logs:PutLogEvents"
        ],
        "Effect": "Allow",
        "Resource": "arn:aws:logs:*:*:*"
      }
    ]
  }

This is the lambda assume policy json -

{
    "Version": "2012-10-17",
    "Statement": [{
        "Action": "sts:AssumeRole",
        "Principal": {
            "Service": "lambda.amazonaws.com"
        },
        "Effect": "Allow",
        "Sid": ""
    }]
}

I have added this to the lambda.tf file -

resource "aws_cloudwatch_log_group" "example" {
  name              = "/test/logs/${var.lambda_function_name}"
}

Although the CloudWatch Log Group '/test/logs/${var.lambda_function_name}' is getting created through terraform, I am unable to write the log of the lambda function to this group.

If I change the lambda policy json to this -

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "Stmt1580204738067",
        "Action": "logs:*",
        "Effect": "Allow",
        "Resource": "*"
    }]
}

Then It automatically stores the log in /aws/lambda/ directory.

How can I make sure that the lambda logs get written into a CloudWatch Log Group that I create and not in the /aws/lambda/ group created by lambda itself?

Upvotes: 13

Views: 17918

Answers (3)

Chalang
Chalang

Reputation: 181

You can follow most if not all of milieere did and it will do it. A quick shortcut you an also do is you can use the AWSLambdaBasicExecution policy that's managed by AWS. This just simplifies Step 2 where you create a policy document. You can define it as a data resource

data "aws_iam_policy" "lambda_basic_execution_policy" {
  arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

You can then use it as an attachment like so

resource "aws_iam_role_policy_attachment" "lambda_flow_log_cloudwatch" {
  role       = aws_iam_role.lambda.name
  policy_arn = data.aws_iam_policy.lambda_basic_execution_policy.arn
}

Just a bit cleaner in my opinion.

Upvotes: 2

milieere
milieere

Reputation: 171

Just adding the log group as a dependency to the lambda is not enough. You also have to attach the IAM policy to the lambda role.

The steps are following:

  1. Define the IAM role for lambda:
resource "aws_iam_role" "iam_for_lambda" {
  name               = "iam_for_lambda"
  assume_role_policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Principal": {
            "Service": "lambda.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
    }]
}
EOF
}
  1. Define the IAM policy that allows lambda to create log streams and put log events
resource "aws_iam_policy" "function_logging_policy" {
  name   = "function-logging-policy"
  policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        Action : [
          "logs:CreateLogStream",
          "logs:PutLogEvents"
        ],
        Effect : "Allow",
        Resource : "arn:aws:logs:*:*:*"
      }
    ]
  })
}
  1. Attach the policy to the IAM role created in step 1, by creating new resource 'aws_iam_role_policy_attachment'
resource "aws_iam_role_policy_attachment" "function_logging_policy_attachment" {
  role       = aws_iam_role.iam_for_lambda.id
  policy_arn = aws_iam_policy.function_logging_policy.arn
}
  1. Define the log group
resource "aws_cloudwatch_log_group" "lambda_log_group" {
  name              = "/aws/lambda/${var.lambda.function_name}"
  retention_in_days = 7
  lifecycle {
    prevent_destroy = false
  }
}
  1. Define your lambda function with the depends_on parameter:
resource "aws_lambda_function" "lambda_function" {
  filename      = "../${var.lambda.function_filename}"
  function_name = "${var.lambda.function_name}"
  role          = aws_iam_role.iam_for_lambda.arn
  handler       = "${var.lambda.handler}"
  layers        = [aws_lambda_layer_version.lambda_layer.arn]
  depends_on    = [aws_cloudwatch_log_group.lambda_log_group]
  source_code_hash = filebase64sha256("../${var.lambda.function_filename}")
  runtime = "python3.9"
}

The IAM policy creation & attachment comes from this article, the rest is from my personal project that worked for me.

Upvotes: 15

Mark B
Mark B

Reputation: 200527

If you want Terraform to manage the CloudWatch log group, you have to create the log group ahead of time with the exact name the Lambda function is going to use for its log group. You can't change the name at all. Then in your Terraform you need to make the log group a dependency of the Lambda function, to make sure Terraform has a chance to create the log group before Lambda creates it automatically.

Upvotes: 5

Related Questions