How to retrieve values (or keys) from maps within maps in Terraform?

Question

Objective:

Create subnets on Azure, for multiple vnets (workloads), from a complex map variable containing other maps originated by the subnet_addrs module

Notes:

• the subnet_addrs module already iterates through a user variable, which is mainly used to assign key names and custom workload attributes

Code:

This:

module "subnet_addrs" {
  #checkov:skip=CKV_TF_1:Todo
  source  = "hashicorp/subnets/cidr"
  version = "1.0.0"
  for_each        = var.workloads
  base_cidr_block = each.value["vnet_prefix"]
  networks = [
    { name = "base", new_bits = 1 },
    { name = "all", new_bits = 1 },
  ]
}

...produces (output "snets" {value = module.subnet_addrs}) this:

  + snets                     = {
      + customerA = {
          + base_cidr_block     = "10.200.17.128/28"
          + network_cidr_blocks = {
              + all  = "10.200.17.136/29"
              + base = "10.200.17.128/29"
            }
          + networks            = [
              + {
                  + cidr_block = "10.200.17.128/29"
                  + name       = "base"
                  + new_bits   = 1
                },
              + {
                  + cidr_block = "10.200.17.136/29"
                  + name       = "all"
                  + new_bits   = 1
                },
            ]
        }
      + customerB = {
          + base_cidr_block     = "10.200.17.192/28"
          + network_cidr_blocks = {
              + all  = "10.200.17.200/29"
              + base = "10.200.17.192/29"
            }
          + networks            = [
              + {
                  + cidr_block = "10.200.17.192/29"
                  + name       = "base"
                  + new_bits   = 1
                },
              + {
                  + cidr_block = "10.200.17.200/29"
                  + name       = "all"
                  + new_bits   = 1
                },
            ]
        }
      + hubA      = {
          + base_cidr_block     = "10.200.17.0/27"
          + network_cidr_blocks = {
              + all  = "10.200.17.16/28"
              + base = "10.200.17.0/28"
            }
          + networks            = [
              + {
                  + cidr_block = "10.200.17.0/28"
                  + name       = "base"
                  + new_bits   = 1
                },
              + {
                  + cidr_block = "10.200.17.16/28"
                  + name       = "all"
                  + new_bits   = 1
                },
            ]
        }
      + hubB      = {
          + base_cidr_block     = "10.200.17.64/27"
          + network_cidr_blocks = {
              + all  = "10.200.17.80/28"
              + base = "10.200.17.64/28"
            }
          + networks            = [
              + {
                  + cidr_block = "10.200.17.64/28"
                  + name       = "base"
                  + new_bits   = 1
                },
              + {
                  + cidr_block = "10.200.17.80/28"
                  + name       = "all"
                  + new_bits   = 1
                },
            ]
        }
    }

Additional comment:

This works fine, but I'd like to move away from creating subnets using a dynamic inside the azurerm_virtual_network resource since service endpoints are now needed:

resource "azurerm_virtual_network" "vnet_workloads" {
  for_each            = var.workloads
  name                = azurecaf_name.caf_rg[each.key].result
  resource_group_name = azurerm_resource_group.rgs_workloads[each.key].name
  location            = azurerm_resource_group.rgs_workloads[each.key].location
  address_space       = [module.subnet_addrs[each.key].base_cidr_block]
  dynamic "subnet" {
    for_each = module.subnet_addrs[each.key].network_cidr_blocks
    content {
      name           = subnet.key # todo: azurecaf
      address_prefix = subnet.value
      security_group = azurerm_network_security_group.nsg_default[each.key].id
    }
  }
}

Issue:

How to make this below work (i.e. how to retrieve the name and address_prefixes attributes from the output of module.subnet_addrs)?

resource "azurerm_subnet" "snets_workloads" {
  for_each             = module.subnet_addrs
  name                 = ???
  resource_group_name  = azurerm_virtual_network.vnet_workloads[each.key].resource_group_name
  virtual_network_name = azurerm_virtual_network.vnet_workloads[each.key].name
  address_prefixes     = ???
  delegation {
    name = "delegation"
    service_delegation {
      name    = "Microsoft.ContainerInstance/containerGroups"
      actions = ["Microsoft.Network/virtualNetworks/subnets/join/action", "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"]
    }
  }
}

Expected result(s):

have azurerm_subnet resource create all the subnets reusing the same name and cidr_block attributes from module.subnet_addrs

Edit (clarification): the target result is the following (e.g.):

customerA (module.subnet_addrs key)

• subnet #1: name = all; cidr_block = 10.200.17.136/29
• subnet #2: name = base; cidr_block = 10.200.17.128/29

customerB

• subnet #1: name = all; cidr_block = ...
• subnet #2: name = base; cidr_block = ... etc.

Any ideas?

---

Additional thoughts:

it looks like my original idea isn't doable since for_each would not accept duplicate keys (customerA, customerA, customerB, etc.). Something like this:

  + snets                     = {
      + customerA = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.128/29"
                  + name       = "base"
                },
            ]
        }
      + customerA = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.136/29"
                  + name       = "all"
                },
            ]
        }
      + customerB = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.192/29"
                  + name       = "base"
                },
            ]
        }
        + customerB = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.200/29"
                  + name       = "all"
                },
            ]
        }
      + hubA      = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.0/28"
                  + name       = "base"
                },
            ]
        }
      + hubA      = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.16/28"
                  + name       = "all"
                },
            ]
        }
      + hubB      = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.64/28"
                  + name       = "base"
                },
            ]
        }
      + hubB      = {
          + networks            = [
              + {
                  + cidr_block = "10.200.17.80/28"
                  + name       = "all"
                },
            ]
        }
    }

I'm thinking about mangling the entire map. For example, having the cidr_block as the new key, and the old key (workload name) as one of its attributes. Something like this:

+ snets                     = {
    + 10.200.17.128/29 = {
        + workload   = "customerA"
        + name       = "base"
    }
    + 10.200.17.136/29 = {
        + workload   = "customerA"
        + name       = "all"
    }
    {and so on...}
}

Is this doable in terraform? transpose maybe? Any suggestions on how to address this?

Answer 1

Answer:

I took the path where the map key gets changed.

Started with a local:

locals {
  snets = flatten([
    for workload_name, workload in module.subnet_addrs : [
      for name, network_cidr_block in workload.network_cidr_blocks : {
        workload_name = workload_name,
        snet_cidr     = network_cidr_block,
        snet_name     = name
      }
    ]
  ])
}

which produced this output:

+ snets_local = [
      + {
          + snet_cidr     = "10.200.17.136/29"
          + snet_name     = "all"
          + workload_name = "customerA"
        },
      + {
          + snet_cidr     = "10.200.17.128/29"
          + snet_name     = "base"
          + workload_name = "customerA"
        },
      + {
          + snet_cidr     = "10.200.17.200/29"
          + snet_name     = "all"
          + workload_name = "customerB"
        },
      + {
          + snet_cidr     = "10.200.17.192/29"
          + snet_name     = "base"
          + workload_name = "customerB"
        },
      + {
          + snet_cidr     = "10.200.17.16/28"
          + snet_name     = "all"
          + workload_name = "hubA"
        },
      + {
          + snet_cidr     = "10.200.17.0/28"
          + snet_name     = "base"
          + workload_name = "hubA"
        },
      + {
          + snet_cidr     = "10.200.17.80/28"
          + snet_name     = "all"
          + workload_name = "hubB"
        },
      + {
          + snet_cidr     = "10.200.17.64/28"
          + snet_name     = "base"
          + workload_name = "hubB"
        },
    ]

And that local led me to this azurerm_subnet resource loop:

resource "azurerm_subnet" "snets_workloads" {
  for_each = {
    for workload in local.snets : workload.snet_cidr => workload
  }
  name                 = azurecaf_name.caf_snet[each.key].result
  resource_group_name  = azurerm_virtual_network.vnet_workloads[each.value["workload_name"]].resource_group_name
  virtual_network_name = azurerm_virtual_network.vnet_workloads[each.value["workload_name"]].name
  address_prefixes     = [each.key]
  delegation {
    name = "delegation"
    service_delegation {
      name    = "Microsoft.ContainerInstance/containerGroups"
      actions = ["Microsoft.Network/virtualNetworks/subnets/join/action", "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"]
    }
  }
}

Where he above's for_each output was this:

  + snets_local_debug = {
      + "10.200.17.0/28"   = {
          + snet_cidr     = "10.200.17.0/28"
          + snet_name     = "base"
          + workload_name = "hubA"
        }
      + "10.200.17.128/29" = {
          + snet_cidr     = "10.200.17.128/29"
          + snet_name     = "base"
          + workload_name = "customerA"
        }
      + "10.200.17.136/29" = {
          + snet_cidr     = "10.200.17.136/29"
          + snet_name     = "all"
          + workload_name = "customerA"
        }
      + "10.200.17.16/28"  = {
          + snet_cidr     = "10.200.17.16/28"
          + snet_name     = "all"
          + workload_name = "hubA"
        }
      + "10.200.17.192/29" = {
          + snet_cidr     = "10.200.17.192/29"
          + snet_name     = "base"
          + workload_name = "customerB"
        }
      + "10.200.17.200/29" = {
          + snet_cidr     = "10.200.17.200/29"
          + snet_name     = "all"
          + workload_name = "customerB"
        }
      + "10.200.17.64/28"  = {
          + snet_cidr     = "10.200.17.64/28"
          + snet_name     = "base"
          + workload_name = "hubB"
        }
      + "10.200.17.80/28"  = {
          + snet_cidr     = "10.200.17.80/28"
          + snet_name     = "all"
          + workload_name = "hubB"
        }
    }

And the final subnet(s) resource plan output looked like this:

  # azurerm_subnet.snets_workloads["10.200.17.80/28"] will be created
  + resource "azurerm_subnet" "snets_workloads" {
      + address_prefixes                               = [
          + "10.200.17.80/28",
        ]
      + enforce_private_link_endpoint_network_policies = (known after apply)
      + enforce_private_link_service_network_policies  = (known after apply)
      + id                                             = (known after apply)
      + name                                           = "snet-all"
      + private_endpoint_network_policies_enabled      = (known after apply)
      + private_link_service_network_policies_enabled  = (known after apply)
      + resource_group_name                            = "rg-myusername-playground-t-hubb"
      + virtual_network_name                           = "rg-myusername-playground-t-hubb"

      + delegation {
          + name = "delegation"

          + service_delegation {
              + actions = [
                  + "Microsoft.Network/virtualNetworks/subnets/join/action",
                  + "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action",
                ]
              + name    = "Microsoft.ContainerInstance/containerGroups"
            }
        }
    }

Maybe it could be improved (it looks a bit ugly). But it works.

Answer 2

This would do the trick.

resource "azurerm_subnet" "snets_workloads" {
  for_each             = module.subnet_addrs
  name                 = each.key
  address_prefixes     = [
    for network in each.value.networks:
      network.cidr_block
  ]
}

(see list comprehension in terraform: https://developer.hashicorp.com/terraform/language/expressions/for)