CODEWITHSUNDEEP
terraform
DarVar
DarVar

Reputation: 18124

Merge list of objects in terraform

Is there a way to merge the following list of objects

variable "common_variables" {
  type = list(object({ var_name = string, var_value = string, workspace_name = string }))
  default = [
    {
      "var_name"       = "location"
      "var_value"      = "West US"
      "workspace_name" = "env-1"
    }
  ]
}

variable "custom_variables" {
  type = list(object({ var_name = string, var_value = string, workspace_name = string }))
  default = [
    {
      "var_name"       = "location"
      "var_value"      = "West Europe"
      "workspace_name" = "env-1"
    }
  ]
}


locals {

  # custom_variables should replace common_variables
  merged_variables = concat(var.common_variables, var.custom_variables)

}

output "merged_variables" {
  value = local.merged_variables
}

Any custom_variables should replace common_variables that match on workspace_name & var_name.

So the output I want is:

merged_variables = [
  {
    "var_name" = "location"
    "var_value" = "West Europe"
    "workspace_name" = "env-1"
  }
]

Upvotes: 6

Views: 42107

Answers (4)

Y Melo
Y Melo

Reputation: 423

Not sure if it's what you want, but you can use map(string) instead of list(object...)).

Here's how:

variable "custom_variables" {
  type    = map(string)
  default = {}
}
locals{
  common_variables = merge(
    {
      Environment     = local.env
      Deployment_type = "terraform"
      var_value       = "West Europe"
    },
    var.custom_variables
  )
}

Then, if you set the variable in terraform.tfvars or any var file like var_value, it should overwrite your commom_variables or add the new one. For example

custom_variables={
  "var_value" = "West US"
}

That's basically the same thing @martin-atkins said.

Upvotes: 0

Diego Velez
Diego Velez

Reputation: 1893

Use setunion https://www.terraform.io/docs/language/functions/setunion.html

Usage:

locals {
    all_users = setunion(var.restricted_users, var.admin_users)
}

Variable types used for this case

variable "restricted_users" {
  type = set(object({
    email = string
    name  = string
    roles = set(string)
  }))

  default = []
}

variable "admin_users" {
  type = set(object({
    email = string
    name  = string
    roles = set(string)
  }))

  default = []
}

Upvotes: 11

Falk Tandetzky
Falk Tandetzky

Reputation: 6600

Merging the lists

Here is a solution that preserves the order (the essential trick is boldly stolen from Martin's answer)

locals {
  common_variables_map = { for v in var.common_variables : "${v.workspace_name}/${v.var_name}" => v }
  custom_variables_map = { for v in var.custom_variables : "${v.workspace_name}/${v.var_name}" => v }
  common_keys          = [ for v in var.common_variables : "${v.workspace_name}/${v.var_name}" ]
  custom_keys          = [ for v in var.common_variables : "${v.workspace_name}/${v.var_name}" ]
  all_keys             = distinct(concat(common_keys, custom_keys))
  
  merged = [
    for k in local.all_keys:
      contains(common_variables_map, k) ? common_variables_map[k] : custom_variables_map[k]
  ]
}

Restructuring variables

As an alternative you might want to consider restructuring your variables a bit and to provide them through input files. If you provide parameters to terraform like this

terraform apply -var-file="common.tfvars" -var-file="custom.tfvars"

then variables declared in both files will be taken from the file listed last. For this to work you of cause need to break down the single list you have into individual parameters. In addition you might want to use a different file for different cases, e.g. use a different file depending on the workspace (it might be an option to wrap the call to terraform with a batch or shell-script or a make-file to make this more convenient).

Upvotes: 0

Martin Atkins
Martin Atkins

Reputation: 74064

The easiest way to do a merge-and-replace is to work with a map rather than a list, so I think I'd start by projecting the two lists into maps where the keys uniquely identify them by the values you want to use to decide what to override, using for expressions:

locals {
  common_variables_map = { for v in var.common_variables : "${v.workspace_name}/${v.var_name}" => v }
  custom_variables_map = { for v in var.custom_variables : "${v.workspace_name}/${v.var_name}" => v }
}

We can then use the maps with merge to get the overriding behavior you want, and then convert back to a list with values if necessary:

locals {
  merged_variables_map = merge(
    local.common_variables_map,
    local.custom_variables_map,
  )
  merged_variables = toset(values(merged_variables_map))
}

It's important to note that converting to a map like this will lose the original ordering of the given lists, because maps are not ordered. The items after merging may therefore be in a different order than the items in var.common_variables or var.custom_variables, and so I made that explicit by converting the result for local.merged_variables using toset. If you consider that okay because the inputs are considered to be ordered anyway, you could make that more explicit to your module's caller by using a set type instead of a list type for the variables too:

variable "common_variables" {
  type = set(object({ var_name = string, var_value = string, workspace_name = string }))
  default = [
    {
      "var_name"       = "location"
      "var_value"      = "West US"
      "workspace_name" = "env-1"
    }
  ]
}

variable "custom_variables" {
  type = set(object({ var_name = string, var_value = string, workspace_name = string }))
  default = [
    {
      "var_name"       = "location"
      "var_value"      = "West Europe"
      "workspace_name" = "env-1"
    }
  ]
}

Sets are not ordered either, so marking a variable as having a set type lets the caller know that the order of the objects is not significant. Terraform can automatically convert from a list to a set (by discarding the ordering), so this won't change the syntax for using the module.

Upvotes: 4

Related Questions