I need nested for_each loop to add multiple disk within resource block for VM’s

Question

I am working on use case to provision vm with for_each loop iteration in vsphere target. during iteration VM should be launch with disk define corresponding to vm name in variables.tf its getting failed to add multiple disk for single vm during the loop.

please check the below code:

variables.tf

variable "demo" {
 type = map(object({
     ip_address = string
     hostname = string
     subnet = string
     vcpu   = string
     ram    = string
     disks   = list(map(any))
  }))
 default = { 
  demo1 = { 
        ip_address =  "1111.1111.1111.1111"
        hostname   =  "test1"
        subnet     =  "subnet1"
        vcpu       =  "4"
        ram        =  "8"
        disks =  {
          "disk0" = {
              name = "disk0" 
              size =  100
              unit_number = 0
           }
           "disk1" = {
              name = "disk1" 
              size =  100
              unit_number = 1
           }
       }
  },
  demo2 = { 
        ip_address =  "222.222.222.222"
        hostname   =  "test2"
        subnet     =  "subnet2"
        vcpu       =  "4"
        ram        =  "8"
        disks =  {
          "disk0" = {
              name = "disk0" 
              size =  100
              unit_number = 0
           }
           "disk1" = {
              name = "disk1" 
              size =  200
              unit_number = 1
           }
       }
  },
  demo3 = { 
        ip_address =  "333.333.333.333"
        hostname   =  "test3"
        subnet     =  "subnet3"
        vcpu       =  "4"
        ram        =  "8"
        disks =  {
          "disk0" = {
              name = "disk0" 
              size =  100
              unit_number = 0
           }
           "disk1" = {
              name = "disk1" 
              size =  300
              unit_number = 1
           }
       }
  }
  }// close default
}// close main

locals.tf

locals {
  env_provisioning = flatten([
    for vm_name, vm_resources in var.demo : [
      for disk_key, disk in vm_resources.disks : {
        vm_name = vm_name
        disk_key = disk_key
        disk = disk
        ip_address = vm_resources.ip_address
        hostname   =  vm_resources.hostname
        subnet     =  vm_resources.subnet
        vcpu       =  vm_resources.vcpu
        ram        =  vm_resources.ram
        name       =  disk.name
        size       =  disk.size
        unit_number = disk.unit_number
      }
    ]
  ])
}

instance.tf

resource "vsphere_virtual_machine" "this" {
 for_each = {
  for vmdisk in local.env_provisioning: "${vmdisk.vm_name}.${vmdisk.disk}" => vmdisk }
  name             = each.value.hostname
  resource_pool_id = data.vsphere_resource_pool.default.id
  datastore_id     = data.vsphere_datastore.datastore.id
  folder           = data.vsphere_folder.folder.path
  firmware         = "efi"
  num_cpus         = each.value.vcpu
  memory           = each.value.ram
  guest_id         = data.vsphere_virtual_machine.template.guest_id
  network_interface {
    network_id = (each.value.subnet == var.backend_subnet ? data.vsphere_network.backend-subnet1.id : data.vsphere_network.frontend-subnet0.id)
  }
 dynamic "disk" {
   for_each = each.value.disk
    content {
      label = each.value.name
      size = each.value.size
      unit_number = each.value.unit_number
     }
  }
  clone {
    template_uuid = data.vsphere_virtual_machine.template.id

    customize {
      linux_options {
          host_name = each.value.hostname
          domain    = var.internal_domain
      }
      network_interface {
        ipv4_address = (each.value.ip_address)
        ipv4_netmask = (each.value.subnet == var.backend_subnet.name ? var.backend_subnet.cidr : var.frontend_subnet.cidr)
      }
      ipv4_gateway = (each.value.subnet == var.backend_subnet.name ? var.backend_subnet.gateway : var.frontend_subnet.gateway)
      dns_server_list = var.dns
    }
  }
}

Expected Results:

after running above code its iterating only for one disk which disk0, its not adding disk1 which is mention in variables.tf for all vm’s i want for every VM define in variables.tf number of corresponding disk should be also added

Answer 1

Your code gave me a couple of errors...

Just on the variable I get:

This default value is not compatible with the variable's type constraint:
element "demo1": attribute "disks": list of map of any single type required.

To get around that I change the variable type to any and after I get:

│ Error: Invalid template interpolation value
│ 
│   on main.tf line 85, in locals:
│   85:     "${vmdisk.vm_name}.${vmdisk.disk}" => vmdisk
│     ├────────────────
│     │ vmdisk.disk is object with 3 attributes
│ 
│ Cannot include the given value in a string template: string required.

To get around that I changed that interpolation to: "${vmdisk.vm_name}.${vmdisk.disk_key}"

with that I can finally get some output ...

---

to troubleshoot this you need to visualize your data, and we don't need such a large sample in the default to begin with, here is what I would do:

variable "demo" {
  type = any
  default = {
    demo1 = {
      hostname = "test1"
      disks = {
        "disk0" = {
          name = "disk0"
        }
        "disk1" = {
          name = "disk1"
        }
      }
    },
    demo2 = {
      hostname = "test2"
      disks = {
        "disk0" = {
          name = "disk0"
        }
        "disk1" = {
          name = "disk1"
        }
      }
    }
  }
}

locals {
  env_provisioning = flatten([
    for vm_name, vm_resources in var.demo : [
      for disk_key, disk in vm_resources.disks : {
        vm_name     = vm_name
        disk_key    = disk_key
        disk        = disk
        hostname    = vm_resources.hostname
        name        = disk.name
      }
    ]
  ])

  final_data = {
    for vmdisk in local.env_provisioning :
    "${vmdisk.vm_name}.${vmdisk.disk_key}" => vmdisk
  }
}

output "final_data" {
  value = local.final_data
}

If we do a terraform plan on that we get:

Changes to Outputs:
  + final_data = {
      + "demo1.disk0" = {
          + disk     = {
              + name = "disk0"
            }
          + disk_key = "disk0"
          + hostname = "test1"
          + name     = "disk0"
          + vm_name  = "demo1"
        }
      + "demo1.disk1" = {
          + disk     = {
              + name = "disk1"
            }
          + disk_key = "disk1"
          + hostname = "test1"
          + name     = "disk1"
          + vm_name  = "demo1"
        }
      + "demo2.disk0" = {
          + disk     = {
              + name = "disk0"
            }
          + disk_key = "disk0"
          + hostname = "test2"
          + name     = "disk0"
          + vm_name  = "demo2"
        }
      + "demo2.disk1" = {
          + disk     = {
              + name = "disk1"
            }
          + disk_key = "disk1"
          + hostname = "test2"
          + name     = "disk1"
          + vm_name  = "demo2"
        }
    }

...there is only one disk per hostname, there is nothing for the dynamic "disk" to loop over.
Is that the structure you expected?

---

Your code

resource "vsphere_virtual_machine" "this" {
  for_each = {
    for vmdisk in local.env_provisioning:
    "${vmdisk.vm_name}.${vmdisk.disk}" => vmdisk 
  }
  name = each.value.hostname
  ...
  dynamic "disk" {
    for_each = each.value.disk
    content {
      label = each.value.name
      size = each.value.size
      unit_number = each.value.unit_number
    }
  }
  ...
}

Everything in that dynamic block is from the parent for_each.
if you need elements from the dynamic loop you need to use the name.
see the documentation for examples: https://developer.hashicorp.com/terraform/language/expressions/dynamic-blocks

---

Keep It Simple

Your variable "demo" already has the "right" structure I see no need for this intermediary env_provisioning we can use the variable directly, something like this:

resource "vsphere_virtual_machine" "this" {
  for_each = var.demo
  name     = each.value.hostname
  ...
  dynamic "disk" {
    for_each = each.value.disks
    content {
      label = disk.value.name
      size  = disk.value.size
      ...
    }
  }
  ...
}