Reputation: 2911
How to for_each through a list(objects) in Terraform 0.12
I need to deploy a list of GCP compute instances. How do I loop for_each through the "vms" in a list of objects like this:
"gcp_zone": "us-central1-a",
"image_name": "centos-cloud/centos-7",
"vms": [
{
"hostname": "test1-srfe",
"cpu": 1,
"ram": 4,
"hdd": 15,
"log_drive": 300,
"template": "Template-New",
"service_types": [
"sql",
"db01",
"db02"
]
},
{
"hostname": "test1-second",
"cpu": 1,
"ram": 4,
"hdd": 15,
"template": "APPs-Template",
"service_types": [
"configs"
]
}
]
}
Upvotes: 181
Views: 340256
Answers (9)
Reputation: 14749
I work a lot with iterators in Terraform, they always gave me bad headaches. Therefore I identified five of the most common iterator patterns (code examples are given below), which helped me construct a lot of nice modules (source).
- Using for_each on a list of strings
- Using for_each on a list of objects
- Using for_each to get the cartasian product of two lists
- Using for_each in a nested block
- Using for_each as a conditional
Using for_each and a list of strings is the easiest to understand, you can always use the toset() function. When working with a list of objects you need to convert it to a map where the key is a unique value. The alternative is to put a map inside your Terraform configuration. Personally, I think it looks cleaner to have a list of objects instead of a map in your configuration. The key usually doesn't have a purpose other than to identify unique items in a map, which can thus be constructed dynamically. I also use iterators to conditionally deploy a resource or resource block, especially when constructing more complex modules.
1. Using for_each on a list of strings
locals {
ip_addresses = ["10.0.0.1", "10.0.0.2"]
}
resource "example" "example" {
for_each = toset(local.ip_addresses)
ip_address = each.key
}
2. Using for_each on a list of objects
locals {
virtual_machines = [
{
ip_address = "10.0.0.1"
name = "vm-1"
},
{
ip_address = "10.0.0.1"
name = "vm-2"
}
]
}
resource "example" "example" {
for_each = {
for index, vm in local.virtual_machines:
vm.name => vm # Perfect, since VM names also need to be unique
# OR: index => vm (unique but not perfect, since index will change frequently)
# OR: uuid() => vm (do NOT do this! gets recreated everytime)
}
name = each.value.name
ip_address = each.value.ip_address
}
3. Using for_each to get the cartesian product of two lists
locals {
domains = [
"https://example.com",
"https://stackoverflow.com"
]
paths = [
"/one",
"/two",
"/three"
]
}
# Loop over both lists and flatten the result
resource "example" "example" {
urls = flatten([
for domain in local.domains : [
for path in local.paths : "${domain}${path}"
]
])
}
# Alternatively, using the setproduct and join functions is cleaner
resource "example" "example" {
urls = [for url in setproduct(local.domains, local.paths): join("", url)]
}
4. Using for_each on a nested block
# Using the optional() keyword makes fields null if not present
variable "routes" {
type = list(
name = string
path = string
config = optional(object({
cache_enabled = bool
https_only = bool
}))
default = []
}
resource "example" "example" {
name = ...
dynamic "route" {
for_each = {
for route in var.routes :
route.name => route
}
content {
# Note: <top_level_block>.value.<object_key>
name = route.value.name
}
dynamic "configuration" {
# Note: <top_level_block>.value.<optional_object_key>
for_each = route.value.config != null ? [1] : []
content {
cache_enabled = route.value.config.cache_enabled
https_only = route.value.config.https_only
}
}
}
5. Using for_each as a conditional (particularly for dynamic blocks)
variable "deploy_example" {
type = bool
description = "Indicates whether to deploy something."
default = true
}
# Using count and a conditional, for_each is also possible here.
# See the next solution using a for_each with a conditional.
resource "example" "example" {
count = var.deploy_example ? 0 : 1
name = ...
ip_address = ...
}
variable "enable_logs" {
type = bool
description = "Indicates whether to enable something."
default = false
}
resource "example" "example" {
name = ...
ip_address = ...
# Note: dynamic blocks cannot use count!
# Using for_each with an empty list and list(1) as a readable alternative.
dynamic "logs" {
for_each = var.enable_logs ? [] : [1]
content {
name = "logging"
}
}
}
Upvotes: 313
Reputation: 975
I've kept together some snippets for remembering how to do do various list comprehension and looping in TF. Some examples, e.g. if given a users = { "foo": { a = true, b = []}} in a tfvar file.
Make from map into a list:
output "reminderHowToListComprehensionEach" {
value = [for e in var.users : e]
}
Keep as a map:
output "reminderHowToListComprehensionKeyValue" {
value = {for k,v in var.users : k => v}
}
Another way
output "reminderHowToListComprehensionKey" {
value = [for k,v in var.users : k]
}
And more useful probably, filtering the map:
output "reminderHowToListComprehensionValueWithConditional" {
value = [for k,v in var.users : v if !v["xyz_bool_attribute"]]
}
See also:
- https://www.terraform.io/docs/language/expressions/for.html
- https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9
- complicated nested looping https://github.com/hashicorp/terraform/issues/20288
Upvotes: 0
Reputation: 271
Yes this is possible, you need to use the for expression in Terraform to achieve this though, the for loop converts the list of objects into a value in which Terraform can loop over using for_each, without the for expression, Terraform cannot loop over the list of objects because there is no key value for Terraform to reference.
Below is a is a simple example:
# variables.tf
variable "nsg_rules" {
description = "list of maps consisting of nsg rules"
type = list(object({
access = string
destination_address_prefixes = list(string)
destination_port_ranges = list(string)
direction = string
name = string
priority = number
protocol = string
source_address_prefixes = list(string)
source_port_range = string
}))
default = [
{
access = "Deny"
destination_address_prefixes = ["10.10.1.0/24", "10.10.2.0/24"]
destination_port_ranges = ["80"]
direction = "Inbound"
name = "DenyHTTPInbound"
priority = 100
protocol = "*"
source_address_prefixes = ["10.0.0.0/24"]
source_port_range = "*"
},
{
access = "Deny"
destination_address_prefixes = ["10.10.10.0/24", "10.10.11.0/24"]
destination_port_ranges = ["22"]
direction = "Inbound"
name = "DenySSHInbound"
priority = 200
protocol = "*"
source_address_prefixes = ["10.0.0.0/24"]
source_port_range = "*"
}
]
}
Use the for expression wrapped in curl brackets to convert the variable value, each maps key will be given the value of each maps name input, for example the first map would be given a key of "DenyHTTPInbound"
resource "azurerm_network_security_rule" "nsg_rules" {
for_each = { for rule in var.nsg_rules : rule.name => rule }
access = each.value.access
destination_address_prefixes = each.value.destination_address_prefixes
destination_port_ranges = each.value.destination_port_ranges
direction = each.value.direction
name = each.value.name
network_security_group_name = azurerm_network_security_group.nsg.name
priority = each.value.priority
protocol = each.value.protocol
resource_group_name = azurerm_resource_group.rg.name
source_address_prefixes = each.value.source_address_prefixes
source_port_range = each.value.source_port_range
}
ref: https://jimferrari.com/2023/02/13/loop-through-list-of-maps-objects-with-terraform/
Upvotes: 2
Reputation: 9487
This is a pretty confusing structure in terraform, but given:
variable services {
type = list(map(string))
description = "services"
default = [
{
name = "abc"
target_port = 9097
health_port = 3780
health_code = 200
protocol = "HTTP"
},
{
name = "def"
target_port = 8580
health_port = 3580
health_code = 200
protocol = "HTTP"
},
{
name = "ghi"
target_port = 80
health_port = 3680
health_code = 200
protocol = "HTTP"
}
]
}
You iterate through resource as so:
resource "aws_lb_listener" "listeners" {
for_each = {
for service in var.services: service.name => service
}
load_balancer_arn = aws_lb.internal.arn
port = each.value.target_port
protocol = each.value.protocol
tags = var.tags
You reference ANOTHER resource which uses a list of objects as so:
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.target_groups[each.value.name].id
}
resource "aws_lb_target_group" "target_groups" {
for_each = {
for service in var.services: service.name => service
}
Notice since aws_lb_target_group is also using an array of maps, you must specify the map property when referencing from another resource, as shown above! That could trip people up.
And if you want to output the list of objects, you do as so:
output "alb_listener_arns" {
value = values(aws_lb_listener.listeners)[*].arn
}
output "target_group_ids" {
value = values(aws_lb_target_group.target_groups)[*].id
}
Upvotes: 1
Reputation: 699
From Terraform 1.3, you can use the for_each and objects with modules like the following:
modules/google_compute_instance/variables.tf
variable "hosts" {
type = map(object({
cpu = optional(number, 1)
ram = optional(number, 4)
hdd = optional(number, 15)
log_drive = optional(number, 300)
template = optional(string, "Template-New")
service_types = list(string)
}))
}
modules/google_compute_instance/main.tf
resource "google_compute_instance" "gcp_instance" {
for_each = {
for key, value in var.hosts :
key => value
}
hostname = each.key
cpu = each.value.cpu
ram = each.value.ram
hdd = each.value.hdd
log_drive = each.value.log_drive
template = each.value.template
service_types = each.value.service_types
}
servers.tf
module "gcp_instances" {
source = "./modules/google_compute_instance"
hosts = {
"test1-srfe" = {
hdd = 20,
log_drive = 500,
service_types = ["sql", "db01", "db02"]
},
"test1-second" = {
cpu = 2,
ram = 8,
template = "APPs-Template",
service_types = ["configs"]
},
}
}
Of course, you can add as many variables as needed and use them in the module.
Upvotes: 51
Reputation: 77
I took reference from the for_each example above and used below. This did not work for me, link below has details. Terraform for_each on custom registry
module "az"{
source="./modules/az"
vpc_id = module.vpc.vpc_id
for_each = toset(keys({for i,v in var.az_sub: i => v}))
availability_zone = var.az_sub[each.value]["az"]
public_cidr_block = var.az_sub[each.value]["public_cidr_block"]
private_cidr_block =var.az_sub[each.value]["private_cidr_block"]
}
Error:module.az is object with 2 attributes If I replace for_each with actual values, the module is working perfectly.
Upvotes: 0
Reputation: 1475
You can do the following:
for_each = toset(keys({for i, r in var.vms: i => r}))
cpu = var.vms[each.value]["cpu"]
Assuming you had the following:
variable "vms" {
type = list(object({
hostname = string
cpu = number
ram = number
hdd = number
log_drive = number
template = string
service_types = list(string)
}))
default = [
{
cpu: 1
...
}
]
}
Upvotes: 10
Reputation: 2911
Seem's like I found what to do. If you pass not the maps of maps but the list of maps you can use such code
resource "google_compute_instance" "node" {
for_each = {for vm in var.vms: vm.hostname => vm}
name = "${each.value.hostname}"
machine_type = "custom-${each.value.cpu}-${each.value.ram*1024}"
zone = "${var.gcp_zone}"
boot_disk {
initialize_params {
image = "${var.image_name}"
size = "${each.value.hdd}"
}
}
network_interface {
network = "${var.network}"
}
metadata = {
env_id = "${var.env_id}"
service_types = "${join(",",each.value.service_types)}"
}
}
It will create actual number of instance and when you remove for example middle one of three(if you create three:)), terraform will remove what we asked.
Upvotes: 110
Reputation: 728
Using the for_each block is pretty new and there's not too much documentation. Some of the best info comes from their announcement blog post: https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each/
Also make sure to check out the Dynamic Blocks section of their documentation: https://www.terraform.io/docs/configuration/expressions.html#dynamic-blocks
From what your example looks like you need to have a set of values for each instance that is created so you'll have a map of maps:
Below is an example I created using Terraform 0.12.12:
variable "hostnames" {
default = {
"one" = {
"name" = "one",
"machine" = "n1-standard-1",
"os" = "projects/coreos-cloud/global/images/coreos-stable-2247-5-0-v20191016",
"zone" = "us-central1-a"
},
"two" = {
"name" = "two",
"machine" = "n1-standard-2",
"os" = "projects/centos-cloud/global/images/centos-8-v20191018",
"zone" = "us-central1-b"
}
}
}
resource "google_compute_instance" "default" {
for_each = var.hostnames
name = each.value.name
machine_type = each.value.machine
zone = each.value.zone
boot_disk {
initialize_params {
image = each.value.os
}
}
scratch_disk {
}
network_interface {
network = "default"
}
}
Terraform plan output:
Terraform will perform the following actions:
# google_compute_instance.default["one"] will be created
+ resource "google_compute_instance" "default" {
+ can_ip_forward = false
+ cpu_platform = (known after apply)
+ deletion_protection = false
+ guest_accelerator = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ label_fingerprint = (known after apply)
+ machine_type = "n1-standard-1"
+ metadata_fingerprint = (known after apply)
+ name = "one"
+ project = (known after apply)
+ self_link = (known after apply)
+ tags_fingerprint = (known after apply)
+ zone = "us-central1-a"
+ boot_disk {
+ auto_delete = true
+ device_name = (known after apply)
+ disk_encryption_key_sha256 = (known after apply)
+ kms_key_self_link = (known after apply)
+ mode = "READ_WRITE"
+ source = (known after apply)
+ initialize_params {
+ image = "projects/coreos-cloud/global/images/coreos-stable-2247-5-0-v20191016"
+ labels = (known after apply)
+ size = (known after apply)
+ type = (known after apply)
}
}
+ network_interface {
+ address = (known after apply)
+ name = (known after apply)
+ network = "default"
+ network_ip = (known after apply)
+ subnetwork = (known after apply)
+ subnetwork_project = (known after apply)
}
+ scheduling {
+ automatic_restart = (known after apply)
+ on_host_maintenance = (known after apply)
+ preemptible = (known after apply)
+ node_affinities {
+ key = (known after apply)
+ operator = (known after apply)
+ values = (known after apply)
}
}
+ scratch_disk {
+ interface = "SCSI"
}
}
# google_compute_instance.default["two"] will be created
+ resource "google_compute_instance" "default" {
+ can_ip_forward = false
+ cpu_platform = (known after apply)
+ deletion_protection = false
+ guest_accelerator = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ label_fingerprint = (known after apply)
+ machine_type = "n1-standard-2"
+ metadata_fingerprint = (known after apply)
+ name = "two"
+ project = (known after apply)
+ self_link = (known after apply)
+ tags_fingerprint = (known after apply)
+ zone = "us-central1-b"
+ boot_disk {
+ auto_delete = true
+ device_name = (known after apply)
+ disk_encryption_key_sha256 = (known after apply)
+ kms_key_self_link = (known after apply)
+ mode = "READ_WRITE"
+ source = (known after apply)
+ initialize_params {
+ image = "projects/centos-cloud/global/images/centos-8-v20191018"
+ labels = (known after apply)
+ size = (known after apply)
+ type = (known after apply)
}
}
+ network_interface {
+ address = (known after apply)
+ name = (known after apply)
+ network = "default"
+ network_ip = (known after apply)
+ subnetwork = (known after apply)
+ subnetwork_project = (known after apply)
}
+ scheduling {
+ automatic_restart = (known after apply)
+ on_host_maintenance = (known after apply)
+ preemptible = (known after apply)
+ node_affinities {
+ key = (known after apply)
+ operator = (known after apply)
+ values = (known after apply)
}
}
+ scratch_disk {
+ interface = "SCSI"
}
}
Plan: 2 to add, 0 to change, 0 to destroy.
Upvotes: 7
Related Questions
- Terraform for-each with list of objects
- Accessing for_each variables of type list(object) from a module
- Looping through list of objects - Terraform
- How to for_each through a list(objects) in Terraform
- Terraform for_each issue with data type
- Terraform for_each not recognizing all values in variable
- Looping over map variable using for_each expression in terraform
- How do I use a for loop to emit a list of objects in Terraform?
- for_each through a object in Terraform 0.12
- Iterating through a list while already using for_each (Terraform 0.12)