Terraform AWS CloudTrail configurations fails

Question

I'm trying to configure AWS CloudTrail using terraform, but still failing on CloudWatch integration. Does anybody see a mistake somewhere?

Terraform CLI and Terraform AWS Provider Version

Terraform v0.13.5

• provider registry.terraform.io/hashicorp/aws v3.15.0

Affected Resource(s)

• aws_cloudtrail

Terraform Configuration Files

# Mendatory VARs
env
aws_region
access_key
secret_key

# Configure the AWS Provider
provider "aws" {
  version = ">= 3.7.0"
  region = var.aws_region
  access_key = var.access_key
  secret_key = var.secret_key
}

# Terraform backend to store current running configuration
terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "some_org"
    workspaces {
      name = "some_workspace"
    }
  }
}

resource "aws_cloudwatch_log_group" "backuping_cloudwatch_log_group" {
  name = "backuping-${var.env}-cloudwatch-log_group"

  tags = {
    Name = "Cloudwatch for backuping CloudTrail"
    Environment = var.env
  }

  depends_on = [aws_s3_bucket.bucket]
}

resource "aws_cloudwatch_log_stream" "backuping_cloudwatch_log_stream" {
  log_group_name = aws_cloudwatch_log_group.backuping_cloudwatch_log_group.id
  name = "backuping-${var.env}-cloudwatch-log_stream"

  depends_on = [aws_cloudwatch_log_group.backuping_cloudwatch_log_group]
}

resource "aws_iam_policy" "backuping_cloudtrail_cloudwatch_policy" {
  name        = "backuping-${var.env}-cloudtrail_cloudwatch_policy"
  description = "Policy to enable ClodTrail logging into CloudWatch on ${var.env}"

  policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AWSCloudTrailCreateLogStream2014110${var.env}",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogStream"
      ],
      "Resource": [
        "${aws_cloudwatch_log_stream.backuping_cloudwatch_log_stream.arn}*"
      ]
    },
    {
      "Sid": "AWSCloudTrailPutLogEvents20141101${var.env}",
      "Effect": "Allow",
      "Action": [
        "logs:PutLogEvents"
      ],
      "Resource": [
        "${aws_cloudwatch_log_stream.backuping_cloudwatch_log_stream.arn}*"
      ]
    }
  ]
}
POLICY

  depends_on = [aws_cloudwatch_log_stream.backuping_cloudwatch_log_stream]
}

resource "aws_iam_role" "backuping_cloudtrail_cloudwatch_role" {
  name = "backuping-${var.env}-cloudtrail_cloudwatch_role"
  path = "/service-role/"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudtrail.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

  tags = {
    Name = "IAM Role for CloudTrail logging into CloudWatch"
    Environment = var.env
  }

  depends_on = [aws_iam_policy.backuping_cloudtrail_cloudwatch_policy]
}

resource "aws_iam_role_policy_attachment" "backuping_cloudtrail_cloudwatch_role_policy_attachement" {
  role       = aws_iam_role.backuping_cloudtrail_cloudwatch_role.name
  policy_arn = aws_iam_policy.backuping_cloudtrail_cloudwatch_policy.arn

  depends_on = [aws_iam_role.backuping_cloudtrail_cloudwatch_role]
}

data "aws_caller_identity" "current" {}

locals {
  backuping_logs_bucket_name = "backuping-${var.env}-logs"
}

resource "aws_s3_bucket" "backuping_logs_bucket" {
  bucket = local.backuping_logs_bucket_name
  acl    = "private"
  force_destroy = true

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        sse_algorithm     = "AES256"
      }
    }
  }

  policy = <<POLICY
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AWSCloudTrailAclCheck",
            "Effect": "Allow",
            "Principal": {
              "Service": "cloudtrail.amazonaws.com"
            },
            "Action": "s3:GetBucketAcl",
            "Resource": "arn:aws:s3:::${local.backuping_logs_bucket_name}"
        },
        {
            "Sid": "AWSCloudTrailWrite",
            "Effect": "Allow",
            "Principal": {
              "Service": "cloudtrail.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::${local.backuping_logs_bucket_name}/AWSLogs/${data.aws_caller_identity.current.account_id}/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        }
    ]
}
POLICY

  tags = {
    Name = "Bucket for backuping logs"
    Environment = var.env
  }
}

resource "aws_s3_bucket_public_access_block" "s3_logs_bucket_public_access" {
  bucket = aws_s3_bucket.backuping_logs_bucket.id

  block_public_acls = true
  block_public_policy = true
  ignore_public_acls = true
  restrict_public_buckets = true
}


resource "aws_cloudtrail" "backuping_cloudtrail" {
  name = "backuping-${var.env}-cloudtrail"
  s3_bucket_name = aws_s3_bucket.backuping_logs_bucket.id
  is_multi_region_trail = true
  enable_log_file_validation = true

  event_selector {
    read_write_type           = "All"
    include_management_events = false

    data_resource {
      type = "AWS::S3::Object"

      # Make sure to append a trailing '/' to your ARN if you want
      # to monitor all objects in a bucket.
      values = ["arn:aws:s3"]

    }
  }

  tags = {
    Name = "CloudTrail for backuping events"
    Environment = var.env
  }

  cloud_watch_logs_role_arn = aws_iam_role.backuping_cloudtrail_cloudwatch_role.arn
  cloud_watch_logs_group_arn = "${aws_cloudwatch_log_group.backuping_cloudwatch_log_group.arn}:*"

  depends_on = [
    aws_iam_role_policy_attachment.backuping_cloudtrail_cloudwatch_role_policy_attachement,
    aws_s3_bucket.backuping_logs_bucket
  ]
}

Debug Output

[DEBUG] plugin.terraform-provider-aws_v3.15.0_x5: 2020/11/17 17:02:38
[DEBUG] [aws-sdk-go] DEBUG: Validate Response cloudtrail/CreateTrail failed, attempt 0/25, error InvalidCloudWatchLogsLogGroupArnException: Access denied. Check the permissions for your role.

Expected Behavior

This should create AWS CloudTrail with logging into S3 bucket and sending logs to CloudWatch

Actual Behavior

S3 bucket is ok, but Cloudwatch adding fails with InvalidCloudWatchLogsLogGroupArnException: Access denied. Check the permissions for your role. When Trail is created without CloudWatch, everything is ok. Also configuring CloudWatch through AWS Console for existing trail is ok. If I compare all perms and roles created through AWS Console with those manually created using Terraform, they are the same, but Terraform ends with error. Also adding CloudWatch to Trail using AWS Console with existing backuping_cloudtrail_cloudwatch_role and existing backuping_cloudwatch_log_group works well. Corresponding policy is automatically created and logging works ok. So all steps except role policy works.

Steps to Reproduce

1. terraform apply

Answer 1

In case you still need to solve this, I have a similar configuration, and had the same issue. The problem is CloudTrail's naming convention. The name of the log stream must be in the format account_ID_CloudTrail_source_region. What I did was to change the name of the log stream to

resource "aws_cloudwatch_log_stream" "test" {
  name           = "${data.aws_caller_identity.current.account_id}_CloudTrail_${data.aws_region.current.name}"
  log_group_name = aws_cloudwatch_log_group.test.name
}

Using the caller_identity and aws_region data sources:

data "aws_region" "current" {}
data "aws_caller_identity" "current" {}

The problem is that CloudTrail forces that conventional name either in the policy (despite our configuration) or when trying to find the log stream, and as the names are not equal, then you get the InvalidCloudWatchLogsLogGroupArnException error.