Terraform Error: Provider produced inconsistent final plan - value "known after apply" causes empty list on plan

Question

The following contrived example causes "Error: Provider produced inconsistent final plan" because of the locals.project_id used in the list of rrdatas on the google_dns_record_set.cdn_dns_txt_record_firebase resource. The project_id value is known only after apply and I do not know how to manage this for the rrdatas list. When I come to apply the plan, the value changes and causes the error mentioned. Your help would be really appreciated.

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 4.24.0"
    }

    random = {
      version = ">= 3.3.2"
    }
  }
}

locals {
  project_id = random_id.project_id.hex
}

resource "random_id" "project_id" {
  keepers = {
    project_id = "my-project-id"
  }
  byte_length = 8
  prefix      = "project-"
}

resource "google_project" "my_project" {
  name       = "A Great Project"
  project_id = random_id.project_id.hex
}

resource "google_dns_record_set" "cdn_dns_txt_record_firebase" {
  name         = "www.bob.com"
  project      = google_project.my_project.project_id
  managed_zone = "bob.com."
  type         = "TXT"
  ttl          = 300
  rrdatas      = [
                  "\"v=spf1 include:_spf.firebasemail.com ~all\"",
                  "firebase=${local.project_id}"
                 ]
}

The plan for the google_dns_record_set.cdn_dns_txt_record_firebase resource looks like this:

  # google_dns_record_set.cdn_dns_txt_record_firebase will be created
  + resource "google_dns_record_set" "cdn_dns_txt_record_firebase" {
      + id           = (known after apply)
      + managed_zone = "bob.com."
      + name         = "www.bob.com"
      + project      = (known after apply)
      + ttl          = 300
      + type         = "TXT"
    }

But I would expect something more like:

  # google_dns_record_set.cdn_dns_txt_record_firebase will be created
  + resource "google_dns_record_set" "cdn_dns_txt_record_firebase" {
      + id           = (known after apply)
      + managed_zone = "bob.com."
      + name         = "www.bob.com"
      + project      = (known after apply)
      + rrdatas      = [
          + "\"v=spf1 include:_spf.firebasemail.com ~all\"",
          + "firebase=(known after apply)",
        ]
      + ttl          = 300
      + type         = "TXT"
    }

Answer 1

Ran into this issue with the AWS Provider. I know it's not quite the same but my solution was to modify the infrastructure via AWS CLI directly. From there I removed the state record of the specific resource in our state store (Terraform Cloud) and then did an import of the resource from AWS.

If you are managing remote state yourself you could likely run the terraform plan and then discard the run, but modify the remote state directly with the changes that Terraform detected. It's definitely a provider bug but that might be a workaround.

Answer 2

You have to check the tfstate file in order to find the incoherence between your terraform code and tfstate attribute. It helped me to fix issue: tfstate

{
      "module": "module.gitlab_cloud_sql",
      "mode": "managed",
      "type": "random_string",
      "name": "random",
      "provider": "provider[\"registry.terraform.io/hashicorp/random\"]",
      "instances": [
        {
          "schema_version": 2,
          "attributes": {
            "id": "pemz",
            "keepers": null,
            "length": null,
            "lower": null,
            "min_lower": null
            "min_numeric": null,
            "min_special": null,
            "min_upper": null,
            "number": null,
            "numeric": null,
            "override_special": null,
            "result": "pemz",
            "special": null,
            "upper": null
          },
          "private": "eyJzY2hlbWFfdmVyc2lvbiI6IjIifQ=="
        }
      ]
    },

Terraform code:

resource "random_string" "random" {
  length    = 4
  special   = false
  lower     = true
  upper     = false
  numeric   = false
  min_upper = 0
  lifecycle {
    ignore_changes = all
  }
}

After fixing the tfstate, it works

{
      "module": "module.gitlab_cloud_sql",
      "mode": "managed",
      "type": "random_string",
      "name": "random",
      "provider": "provider[\"registry.terraform.io/hashicorp/random\"]",
      "instances": [
        {
          "schema_version": 2,
          "attributes": {
            "id": "pemz",
            "keepers": null,
            "length": 4,
            "lower": true,
            "min_lower": 0,
            "min_numeric": 0,
            "min_special": 0,
            "min_upper": 0,
            "number": false,
            "numeric": false,
            "override_special": null,
            "result": "pemz",
            "special": false,
            "upper": false
          },
        }
      ]
    },

Answer 3

We can see from the plan information that the provider has indeed generated an invalid plan in this case, for the reason you observed: you set rrdatas in the configuration, and so the provider ought to have generated an initial plan to set those values.

As is being discussed over in the bug report you filed about this, the provider seems to be mishandling the unknown value you passed here, and returning a plan that has it set to null instead of to unknown as expected.

Until that bug is fixed in the provider, I think the main workaround would be to find some way to ensure that Terraform Core already knows the value of local.project_id before asking the provider to plan resource "google_dns_record_set" "cdn_dns_txt_record_firebase".

One way to achieve that would be to create a targeted plan that tells Terraform to focus only on generating that random ID in its first operation, and then once that's succeeded you can use normal Terraform applies moving forward as long as you avoid regenerating that random ID:

• terraform apply -target=random_id.project_id to just generate that random ID, without planning anything else.
• terraform apply to converge everything else. This should succeed because random_id.project_id.hex should already be known from the previous run, and so local.project_id will be known too. The provider then won't run into this buggy behavior, because it will see rrdatas as being a known list of strings rather than as an unknown value.