is a list of string, known only after apply when for_each involved

Question

I am using the VPC module to create a VPC and subnets.

Once the subnets are created, I want to share them with other accounts. The module works perfectly fine and creates all the subnets. I need the subnet IDs so that I can then use RAM to share the subnets.

My code roughly looks like

# Create VPC and subnets
module "vpc" {
...
...
}

# Next get subnet IDs
data "aws_subnets" "dev_subnet" {
  filter {
    name   = "vpc-id"
    values = [module.vpc.vpc_id]
  }

  tags = {
    Environment = "pe-dev*"
  }
}


# Create resource share and principal association
resource "aws_ram_resource_share" "share_subnets_with_dev_account" {}
resource "aws_ram_principal_association" "share_subnets_with_dev_account" {}

Now from the subnet IDs I need to extract the ARNs and then make a resource association

resource "aws_ram_resource_association" "example" {
  for_each = toset(data.aws_subnets.dev_subnet.ids)

  resource_arn       =  "arn:aws:ec2:${var.region}:${var.aws_account_id}:subnet/${each.value}"
  resource_share_arn = aws_ram_resource_share.share_subnets_with_dev_account.arn

}

But when I do a fresh terrafrom apply I get the error

│ Error: Invalid for_each argument
│
│   on main.tf line 110, in resource "aws_ram_resource_association" "example":
│  110:   for_each = toset(data.aws_subnets.dev_subnet.ids)
│     ├────────────────
│     │ data.aws_subnets.dev_subnet.ids is a list of string, known only after apply
│
│ The "for_each" set includes values derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this resource.
│
│ When working with unknown values in for_each, it's better to use a map value where the keys are defined statically in your configuration and where only the values contain apply-time results.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.

What came to my mind was to add a depends_on. Something like this

resource "aws_ram_resource_association" "example" {
  for_each = toset(data.aws_subnets.dev_subnet.ids)

  resource_arn       =  "arn:aws:ec2:${var.region}:${var.aws_account_id}:subnet/${each.value}"
  resource_share_arn = aws_ram_resource_share.share_subnets_with_dev_account.arn

  depends_on = [
    module.vpc.aws_subnet.private
  ]

but now i get

│ Error: Invalid depends_on reference
│
│   on main.tf line 116, in resource "aws_ram_resource_association" "example":
│  116:     module.vpc.aws_subnet.private
│
│ References in depends_on must be to a whole object (resource, etc), not to an attribute of an object.

Any idea how I can wait for the subnets to be created and get subnet IDs before aws_ram_resource_association is created ?

EDIT:

What was running

data "aws_subnets" "dev_subnet" {
  filter {
    name   = "vpc-id"
    values = [module.vpc.vpc_id]
  }

  tags = {
    Environment = "dev-*"
  }
}

data "aws_subnet" "dev_subnet" {
  for_each = toset(data.aws_subnets.dev_subnet.ids)
  id       = each.value
}

output "dev_subnet_arns" {
  value = [for s in data.aws_subnet.dev_subnet : s.arn]
}

Result

  + dev_subnet_arns = [
      + "arn:aws:ec2:ca-central-1:0097747:subnet/subnet-013987fd9651c3545",
      + "arn:aws:ec2:ca-central-1:0477747:subnet/subnet-015d76b264280321a",
      + "arn:aws:ec2:ca-central-1:0091747:subnet/subnet-026cd0402fe283c33",
    ]

but only when i do a tf plan after a previosuly run tf apply.

IF I do a tf destroy and recreate everything then i get the error again

tf plan
╷
│ Error: Invalid for_each argument
│
│   on main.tf line 116, in data "aws_subnet" "dev_subnet":
│  116:   for_each = toset(data.aws_subnets.dev_subnet.ids)
│     ├────────────────
│     │ data.aws_subnets.dev_subnet.ids is a list of string, known only after apply
│
│ The "for_each" set includes values derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this resource.
│
│ When working with unknown values in for_each, it's better to use a map value where the keys are defined statically in your configuration and where only the values contain apply-time results.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.

Answer 1

The key problem here is that for_each must be evaluated before the resource is planned, rather than before the resource is created.

The result of data.aws_subnets.dev_subnet.ids depends on the VPC ID, and the VPC ID can't be known until the VPC has been created. But aws_ram_resource_association.example must also be planned at the same time, before the VPC has been created, and so the only way for Terraform to resolve this would be to create the VPC during the planning step, and that would violate the expectation that Terraform doesn't perform any actions until the apply step.

---

With the architecture you have here, where the calling module is trying to retrieve a set of subnets that haven't been created yet (because their containing VPC also hasn't been created yet), the only way to resolve this would be to first run Terraform with the extra option -target, to force it to create the VPC and subnets first before planning anything else:

• terraform apply -target=module.vpc first, which will cause Terraform to create and apply a partial plan only including the resources declared in that module and whatever they depend on.
• terraform apply with no arguments afterwards, to plan and apply everything else that the partial plan didn't include.

You can then use terraform apply as normal for ongoing maintenence, as long as you never replace the VPC and thereby cause its ID to become unknown again.

---

To avoid the need for this extra special bootstrapping step, the better design would be for the VPC module to export the subnets it declares as an additional output value, which means that Terraform can use the set of subnets that are planned for creation, rather than the set of subnets that already exist.

Unfortunately this VPC module you are using doesn't export the subnet IDs in a way that's suitable for use with for_each: it only exports the subnet IDs alone, without associating them with a unique key that can identify them during planning. Therefore unfortunately with this module as currently designed you'll need to use count instead of for_each:

resource "aws_ram_resource_association" "example" {
  count = length(module.vpc.private_subnets)

  resource_arn       =  "arn:aws:ec2:${var.region}:${var.aws_account_id}:subnet/${module.vpc.private_subnets[count.index]}"
  resource_share_arn = aws_ram_resource_share.share_subnets_with_dev_account.arn
}

This will cause the instances of this resource to be tracked by their position in the list of subnets, and so if you add or remove subnets in future their associations with the list items will change.

---

To use for_each here would require this module to export the subnets as a mapping where the keys are values that can be determined statically from the configuration -- such as the CIDR blocks -- and the values are the information about each subnet.

Here is a hypothetical output value that the module could include to support this, but to add this will require that you create your own fork of the shared module and modify it:

output "private_subnets" {
  value = {
    for sn in aws_subnet.private : sn.cidr_block => {
      id = sn.id
    }
  }
}

With the module modified in this way, your calling module can then use for_each with this value:

resource "aws_ram_resource_association" "example" {
  for_each = module.vpc.private_subnets

  resource_arn       =  "arn:aws:ec2:${var.region}:${var.aws_account_id}:subnet/${each.value.id}"
  resource_share_arn = aws_ram_resource_share.share_subnets_with_dev_account.arn
}

With this new structure, Terraform will track the instances of aws_ram_resource_association.example by using their CIDR blocks as unique identifiers, and so you can add and remove CIDR blocks over time and Terraform will correctly understand which "RAM Resource Association" belongs to which subnet and add/remove the individual ones that correlate.