Terraform flatten and looping issue inside modules

Question

I'm trying to create a module in Terraform for creating Azure resources and facing some issues. This module creates a resource group, subnet, vnet and Role bindings. I see that the below code creates the resources twice because of the loop. Does the for_each loop work in such a way that the entire resource or module block will be executed each time it loops? I'm new to Terraform and come from a Java background. Also, ideally would like to use the flatten inside the module without locals possibly, any way to do that? Code is below.

locals {
  groupsbyrole = flatten([
    for roleName, groupList in var.testproject1_role_assignments : [
      for groupName in groupList : {
        role_name  = roleName
        group_name = groupName
      }
    ]
  ])
}

module "testproject1" {
  source       = "C:\\Users\\ebo1h8h\\Documents\\Project\\Automation\\Terraform\\Code\\Azure\\modules\\sandbox-module"
  short_name = "testproj"
  # Resource Group Variables
  az_rg_location      = "eastus"
  az_tags = {
    Environment   = "Sandbox"
    CostCenter    = "Department"
    ResourceOwner = "Vikram"
    Project       = "testproj"
    Role          = "Resource Group"
  }
  address_space   = ["10.0.0.0/16"]
  subnet_prefixes = ["10.0.1.0/24"]
  subnet_names    = ["a-npe-snet01-sbox"]
  vnet_location   = var.az_rg_location
  for_each = {
    for group in local.groupsbyrole : "${group.role_name}.${group.group_name}}" => group
  }
  principal_id         = each.value.group_name
  role_definition_name = each.value.role_name
}

And here is the role_assignments variable

variable "testproject1_role_assignments" {
  type = map(list(string))
  default = {
    "Contributor" = ["prod-azure-contrib-sbox", "gcp-org-network-engineering"],
    "Owner"       = ["gcp-org-cloud-delivery"]
  }
}

The above code creates 12 resources when it should be only 6. The only was I was able to get around this is have the resource "azurerm_role_assignment" "role_assignment" as a separate module. Ideally, I want to pass the role assignments variable in each of the module to be created so that it creates a set of resources.

Any pointers on how to achieve that?

Thanks,

Answer 1

I think some things got mixed up here. The code you created will only create 3 instances of the module (example 1). Additionally, example 2 shows the scenario where you would have 6 instances.

• The combination of role and groups only for the groups in the list assigned to the role (2 x 1 + 1 x 1) equals 3.

Example 1:

  variable "role_assignments" {
    type = map(list(string))  
    default = {
      Contributor = ["prod-azure-contrib-sbox", "gcp-org-network-engineering"],
      Owner       = ["gcp-org-cloud-delivery"]
    }
  }

locals {
  groups_by_role = [
    for role, groups in var.role_assignments : [
      for group in groups: {
        key   = "${role}.${group}"
        role  = role
        group = group
      }
    ]
  ]
}

output "result" {
  local.groups_by_role
}

# Produces 3 results
result = [
  {
    key   = "Contributor.prod-azure-contrib-sbox"
    group = "prod-azure-contrib-sbox"
    role  = "Contributor"
  },
  {
    key   = "Contributor.gcp-org-network-engineering"
    group = "gcp-org-network-engineering"
    role  = "Contributor"
  },
  {
    key   = "Owner.gcp-org-cloud-delivery"
    group = "gcp-org-cloud-delivery"
    role  = "Owner"
  },
]

• Every combination of role and group (2 x 3) equals 6. Can be done with the help of the setproduct function. This function produces the Cartasian product of two (or more) lists.

Example 2:

variable "role_assignments" {
  type = map(list(string))  
  default = {
    Contributor = ["prod-azure-contrib-sbox", "gcp-org-network-engineering"],
    Owner       = ["gcp-org-cloud-delivery"]
  }
}

locals {
  all_combinations = [
    for item in setproduct(
      keys(var.role_assignments),
      flatten(values(var.role_assignments))
    ) : {
      key   = "${item[0]}.${item[1]}"
      role  = item[0]
      group = item[1]
    }
  ]
}

output “result” {
  local.all_combinations
}

# Produces 6 results
result = [
  {
    key   = "Owner.prod-azure-contrib-sbox"
    group = "prod-azure-contrib-sbox"
    role  = "Owner"
  },
  {
    key   = "Owner.gcp-org-network-engineering"
    group = "gcp-org-network-engineering"
    role  = "Owner"
  },
  {
    key   = "Owner.gcp-org-cloud-delivery"
    group = "gcp-org-cloud-delivery"
    role  = "Owner"
  },
  {
    key   = "Contributor.prod-azure-contrib-sbox"
    group = "prod-azure-contrib-sbox"
    role  = "Contributor"
  },
  {
    key   = "Contributor.gcp-org-network-engineering"
    group = "gcp-org-network-engineering"
    role  = "Contributor"
  },
  {
    key   = "Contributor.gcp-org-cloud-delivery"
    group = "gcp-org-cloud-delivery"
    role  = "Contributor"
  },
]

Answer 2

The docs state

If a resource or module block includes a for_each argument whose value is a map or a set of strings, Terraform will create one instance for each member of that map or set.

So in your scenario you are creating 3 instances of the module, whereas it sounds like you want to pass in the local.groupsbyrole object as a variable in the module and only attach the for_each to the resources you want multiple instances of.

Sidenote: You could simplify your local by adding group like below:

locals {
    groupsbyrole = flatten([
        for roleName, groupList in var.testproject1_role_assignments : [
            for groupName in groupList : {
                role_name  = roleName
                group_name = groupName
                group = "${roleName}.${groupName}"
            }
        ]
    ])
}

Tip: I find adding an output to see the shape of the object whilst developing can also be useful

output "test_output" {
    value = local.groupsbyrole
}

Then when you run plan you will see your object

test_output = [
  + {
      + group      = "Contributor.prod-azure-contrib-sbox"
      + group_name = "prod-azure-contrib-sbox"
      + role_name  = "Contributor"
    },
  + {
      + group      = "Contributor.gcp-org-network-engineering"
      + group_name = "gcp-org-network-engineering"
      + role_name  = "Contributor"
    },
  + {
      + group      = "Owner.gcp-org-cloud-delivery"
      + group_name = "gcp-org-cloud-delivery"
      + role_name  = "Owner"
    },
]