CODEWITHSUNDEEP
terraform
Albert Wolchesky
Albert Wolchesky

Reputation: 135

Terraform: Creating maps with matching key fails with "duplicate object keys"

I am trying to create a map of secondary ranges for the GCP VPC module here and have the following defined in my locals:

  secondary_ranges = {
  for name, config in var.subnet_config : config.subnet_name => [
      {
        range_name    = local.ip_range_pods
        ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.0.0/17"
      },
      {
        range_name    = local.ip_range_services
        ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.128.0/17"
      }
    ]
  }

subnet_config is defined as follows:

subnet_config   = {
    cluster1 = {
        region           = "us-east1"
        subnet_name      = "default"
    },
    cluster2 = {
        region           = "us-west1"
        subnet_name      = "default"
    }
}

This creates the secondary subnets just fine if the subnet names are unique but fails with the error below if the subnet names (which end up being the key values) are not unique:

Two different items produced the key "default" in this 'for' expression. If duplicates are expected, use the ellipsis (...) after the value expression to enable grouping by key.

I'm trying to figure out if I can use grouping mode if the value is a list and if so, how?

Any help would be greatly appreciated.

Upvotes: 2

Views: 8703

Answers (1)

Martin Atkins
Martin Atkins

Reputation: 74564

If you use the grouping mode in this case then it would be to group the outermost for expression, which is producing a map, because that's the one whose keys you'd be grouping by.

We can start by adding the grouping mode modifier to that and see what happens:

 secondary_ranges_pairs = {
   for name, config in var.subnet_config : config.subnet_name => [
     {
       range_name    = local.ip_range_pods
       ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.0.0/17"
     },
     {
       range_name    = local.ip_range_services
       ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.128.0/17"
     }
   ]...
 }

The effect of the expression above would be to create a map of lists of lists of objects, where the deepest lists are each pairs of objects because of how your inner for expression is written.

To turn that into the map of lists of objects which I think you're hoping for, you can then use flatten in a separate step:

  secondary_ranges = {
    for k, pairs in local.secondary_ranges_pairs : k => flatten(pairs)
  }

flatten recursively walks a data structure where there are lists of lists and concatenates all of the nested lists together into a single flat list.

A word of caution: you seem to be using a lexical sort of the subnet_config keys in order to derive network numbering. That means that if you add new elements to your var.subnet_config whose keys sort earlier than any existing ones (for example, if you were to add in a cluster0 into what you showed in your question) then you'll implicitly renumber all of the subsequent networks, which is likely to cause a lot of churn recreating objects, and the change might not even be possible if those networks contain other objects.

I'd typically recommend instead being explicit about what number you've assigned to each network, by including then as part of the var.subnet_config objects. You can then clearly see which numbers you've assigned and make sure that any new networks will always be assigned a later number without disturbing any existing assignments.

There's also an official Terraform module hashicorp/subnets/cidr which aims to encapsulate subnet numbering calculations. The design of that module means that it wouldn't be completely straightforward to adopt it for your use-case (since you're allocating two levels of subnet at once) but it might be useful to study to see whether any of the design tradeoffs made there are relevant to your module.

Upvotes: 1

Related Questions