CODEWITHSUNDEEP
google-cloud-platformterraformterraform-provider-gcp
N. Blore
N. Blore

Reputation: 43

My Terraform Plan always shows changes to my cloud storage buckets even when I don't make changes?

Whenever I run a terraform plan from my root terraform/ folder, everything runs as expected and new changes reflect accordingly. However.. every single time I run a plan, without fail, there are changes to my cloud storage bucket modules even though I have not touched them.

It's always changes to the lifecycle_rule and I could not really see why. Any help appreciated :) My best guess is something to do with null default values? But that isn't causing any issues on any of my other modules so I really have no idea. If any other info needed please let me know. Thank you so much

Terraform Plan: (Just one bucket example, all are exactly like this)

  # module.bronze_dq_0.google_storage_bucket.this will be updated in-place
  ~ resource "google_storage_bucket" "this" {
        id                          = "xx-bucket-name"
        name                        = "xx-bucket-name"
        # (11 unchanged attributes hidden)

      ~ lifecycle_rule {
          - condition {
              - age                        = 0 -> null
              - days_since_custom_time     = 0 -> null
              - days_since_noncurrent_time = 0 -> null
              - matches_prefix             = [] -> null
              - matches_storage_class      = [] -> null
              - matches_suffix             = [] -> null
              - num_newer_versions         = 0 -> null
              - with_state                 = "ANY" -> null
            }
          + condition {
              + matches_prefix        = []
              + matches_storage_class = []
              + matches_suffix        = []
              + with_state            = (known after apply)
            }

            # (1 unchanged block hidden)
        }

        # (1 unchanged block hidden)
    }

I am using terraform modules that I have created to make these buckets. An example is

module "gold_denormalisation" {
  source      = "./modules/CloudStorage"
  location    = local.locations.london
  bucket_name = "xx-bucket-name"

  uniform_bucket_level_access = true
  public_access_prevention    = "enforced"

  lifecycle_rule = [{     
    action = [{
      type = "Delete"
    }]
    condition = [{
      age_days = 180
    }]

    condition = [{
      num_newer_versions = 1
    }]
  }]

  encryption = {
    default_kms_key_name = "projects/${local.project_id}/locations/europe-west2/keyRings/${module.bucket-storage-keys.keyring_name}/cryptoKeys/${module.bucket-storage-keys.key_name}"
  }
}

Module creation on lifecycle_rule..

  dynamic "lifecycle_rule" {
  for_each = var.lifecycle_rule == null ? [] : var.lifecycle_rule
  iterator = rule
  content {
    dynamic "action" {
      for_each = rule.value.action == null ? [] : rule.value.action
      content {
        type          = action.value.type
        storage_class = lookup(action, "storage_class", null)
      }
    }
    dynamic "condition" {
      for_each = rule.value.condition == null ? [] : rule.value.condition
      content {
        age                        = lookup(condition, "age", null)
        created_before             = lookup(condition, "created_before", null)
        with_state                 = lookup(condition, "with_state", null)
        matches_storage_class      = lookup(condition, "matches_storage_class", null)
        matches_prefix             = lookup(condition, "matches_prefix", null)
        matches_suffix             = lookup(condition, "matches_suffix", null)
        num_newer_versions         = lookup(condition, "num_newer_versions", null)
        custom_time_before         = lookup(condition, "custom_time_before", null)
        days_since_custom_time     = lookup(condition, "days_since_custom_time", null)
        days_since_noncurrent_time = lookup(condition, "days_since_noncurrent_time", null)
        noncurrent_time_before     = lookup(condition, "noncurrent_time_before", null)
      }
    }
  }
}

The lifecycle_rule variable..

variable "lifecycle_rule" {
  description = "(Optional) Configuration if the bucket is to have logging enabled or not."
  type = list(object({
    action = list(object({
      type          = string # (Required) The type of the action of this Lifecycle Rule. Supported values include: 'Delete', 'SetStorageClass' and 'AbortIncompleteMultipartUpload'.
      storage_class = optional(string) # (Required if action type is SetStorageClass) The target Storage Class of objects affected by this Lifecycle Rule. Supported values include: 'STANDARD', 'MULTI_REGIONAL', 'REGIONAL', 'NEARLINE', 'COLDLINE', 'ARCHIVE'.
    }))
    condition = list(object({ # The condition block supports the following elements, and requires at least one to be defined. If you specify multiple conditions in a rule, an object has to match all of the conditions for the action to be taken:
      age                        = optional(number) # (Optional) Minimum age of an object in days to satisfy this condition.
      created_before             = optional(string) # (Optional) A date in the RFC 3339 format YYYY-MM-DD. This condition is satisfied when an object is created before midnight of the specified date in UTC.
      with_state                 = optional(string) # (Optional) Match to live and/or archived objects. Unversioned buckets have only live objects. Supported values include: 'LIVE', 'ARCHIVED', 'ANY'.
      matches_storage_class      = optional(string) # (Optional) Storage Class of objects to satisfy this condition. Supported values include: 'STANDARD', 'MULTI_REGIONAL', 'REGIONAL', 'NEARLINE', 'COLDLINE', 'ARCHIVE', 'DURABLE_REDUCED_AVAILABILITY'.
      matches_prefix             = optional(string) # (Optional) One or more matching name prefixes to satisfy this condition.
      matches_suffix             = optional(string) # (Optional) One or more matching name suffixes to satisfy this condition.
      num_newer_versions         = optional(string) # (Optional) Relevant only for versioned objects. The number of newer versions of an object to satisfy this condition.
      custom_time_before         = optional(string) # (Optional) A date in the RFC 3339 format YYYY-MM-DD. This condition is satisfied when the customTime metadata for the object is set to an earlier date than the date used in this lifecycle condition.
      days_since_custom_time     = optional(number) # (Optional) Days since the date set in the customTime metadata for the object. This condition is satisfied when the current date and time is at least the specified number of days after the customTime.
      days_since_noncurrent_time = optional(number) # (Optional) Relevant only for versioned objects. Number of days elapsed since the noncurrent timestamp of an object.
      noncurrent_time_before     = optional(string) # (Optional) Relevant only for versioned objects. The date in RFC 3339 (e.g. 2017-06-13) when the object became nonconcurrent.
    }))
  }))
  default = null
}

Expecting 0 changes to the terraform plan, should be no infrastructure updates

Upvotes: 0

Views: 1101

Answers (1)

OmarR
OmarR

Reputation: 11

I think an answer to make @N.Blore's explanation clearer may help some people here...

(NOTE that this proposed solution was tested with Terraform versions v1.3.9 & v1.6.5 as well as hashicorp/google v5.6.0 & v5.7.0)

A Terraform module is used to create the buckets like this example below...

module "bucket_name_of_bucket" {
      source        = "./modules"
      name          = "name-of-bucket"
      multi_region  = "eu"
      storage_class = "STANDARD"
      versioning    = false

      lifecycle_rule = [
          {
              action = [{
                  type = "Delete",
              }]
              condition = [{
                  created_before = "2022-11-01"
                  matches_suffix = [
                      ".xml"
                  ]
              }]
          },
     ]
}

This then calls a child module that defines the resource (so it can be re-used by multiple calling modules...). Note that there are 2 main differences between this child module & the proposed child module above.

  1. rule.value has to call the value from a key:value pair hence ["some_key"] syntax.
  2. lookup in the dynamic block has to refer to the map it is reading, action.value or condition.value not action nor condition.

Here is the amended child module...

resource "google_storage_bucket" "default" {
  name                        = var.name
  location                    = var.multi_region
  storage_class               = var.storage_class
  uniform_bucket_level_access = true
  public_access_prevention    = "enforced"
  versioning {
    enabled = var.versioning
  }
  dynamic "lifecycle_rule" {
    for_each = var.lifecycle_rule == null ? [] : var.lifecycle_rule
    iterator = rule
    content {
      dynamic "action" {
        for_each = rule.value["action"] == null ? [] : rule.value["action"]
        content {
          type          = action.value["type"]
          storage_class = lookup(action.value, "storage_class", null)
        }
      }
      dynamic "condition" {
        for_each = rule.value["condition"] == null ? [] : rule.value["condition"]
        content {
          age                        = lookup(condition.value, "age", null)
          created_before             = lookup(condition.value, "created_before", null)
          with_state                 = lookup(condition.value, "with_state", "")
          matches_storage_class      = lookup(condition.value, "matches_storage_class", [])
          matches_prefix             = lookup(condition.value, "matches_prefix", [])
          matches_suffix             = lookup(condition.value, "matches_suffix", [])
          num_newer_versions         = lookup(condition.value, "num_newer_versions", null)
          custom_time_before         = lookup(condition.value, "custom_time_before", null)
          days_since_custom_time     = lookup(condition.value, "days_since_custom_time", null)
          days_since_noncurrent_time = lookup(condition.value, "days_since_noncurrent_time", null)
          noncurrent_time_before     = lookup(condition.value, "noncurrent_time_before", null)
        }
      }
    }
  }
}

Finally, the variable definition needed 1 or 2 tweaks as well to avoid data type errors with terraform plan. The condition keys matches_storage_class, matches_prefix & matches_suffix needed to be of type optional(list(string)).

This then gave the below amended variable definition...

variable "lifecycle_rule" {
  description = "(Optional) Configuration if the bucket is to have data lifecycle rules enabled or not."
  type = list(object({
    action = list(object({
      type          = string           # (Required) The type of the action of this Lifecycle Rule. Supported values include: 'Delete', 'SetStorageClass' and 'AbortIncompleteMultipartUpload'.
      storage_class = optional(string) # (Required if action type is SetStorageClass) The target Storage Class of objects affected by this Lifecycle Rule. Supported values include: 'STANDARD', 'MULTI_REGIONAL', 'REGIONAL', 'NEARLINE', 'COLDLINE', 'ARCHIVE'.
    }))
    condition = list(object({                            # The condition block supports the following elements, and requires at least one to be defined. If you specify multiple conditions in a rule, an object has to match all of the conditions for the action to be taken:
      age                        = optional(number)       # Minimum age of an object in days to satisfy this condition.
      created_before             = optional(string)       # A date in the RFC 3339 format YYYY-MM-DD. This condition is satisfied when an object is created before midnight of the specified date in UTC.
      with_state                 = optional(string)       # Match to live and/or archived objects. Non-versioned buckets have only live objects. Supported values include: 'LIVE', 'ARCHIVED', 'ANY'.
      matches_storage_class      = optional(list(string)) # Storage Class of objects to satisfy this condition. Supported values include: 'STANDARD', 'MULTI_REGIONAL', 'REGIONAL', 'NEARLINE', 'COLDLINE', 'ARCHIVE', 'DURABLE_REDUCED_AVAILABILITY'.
      matches_prefix             = optional(list(string)) # One or more matching name prefixes to satisfy this condition.
      matches_suffix             = optional(list(string)) # One or more matching name suffixes to satisfy this condition.
      num_newer_versions         = optional(number)       # Relevant only for versioned objects. The number of newer versions of an object to satisfy this condition.
      custom_time_before         = optional(string)       # A date in the RFC 3339 format YYYY-MM-DD. This condition is satisfied when the customTime metadata for the object is set to an earlier date than the date used in this lifecycle condition.
      days_since_custom_time     = optional(number)       # Days since the date set in the customTime metadata for the object. This condition is satisfied when the current date and time is at least the specified number of days after the customTime.
      days_since_noncurrent_time = optional(number)       # Relevant only for versioned objects. Number of days elapsed since the non-current timestamp of an object.
      noncurrent_time_before     = optional(string)       # Relevant only for versioned objects. The date in RFC 3339 (e.g. 2017-06-13) when the object became non-concurrent.
    }))
  }))
  default = []
}

Upvotes: 1

Related Questions