Invalid for_each argument during the aws_lb_target_group_attachment

Question

I have two target_groups - one for port 80 and another for 443. Also have two instances as the members (of that NLB) and I need to attach both of the target groups to each instance. So this is how I have configured, to attach:

// Creates the target-group
resource "aws_lb_target_group" "nlb_target_groups" {
  for_each = {
    for lx in var.nlb_listeners : "${lx.protocol}:${lx.target_port}" => lx
  }
  name                 = "${var.vpc_names[var.idx]}-tgr-${each.value.target_port}"
  deregistration_delay = var.deregistration_delay
  port                 = each.value.target_port
  protocol             = each.value.protocol
  vpc_id               = var.vpc_ids[var.idx]
  proxy_protocol_v2    = true

  health_check {
    port                = each.value.health_port
    protocol            = each.value.protocol
    interval            = var.health_check_interval
    healthy_threshold   = var.healthy_threshold
    unhealthy_threshold = var.unhealthy_threshold
  }
}

// Attach the target groups to the instance(s)
resource "aws_lb_target_group_attachment" "tgr_attachment" {
  for_each = {
    for pair in setproduct(keys(aws_lb_target_group.nlb_target_groups), var.nlb_members.ids) : "${pair[0]}:${pair[1]}" => {
      target_group = aws_lb_target_group.nlb_target_groups[pair[0]]
      instance_id  = pair[1]
    }
  }
  target_group_arn = each.value.target_group.arn
  target_id        = each.value.instance_id
  port             = each.value.target_group.port
  #target_id       = [for tid in range(var.inst_count) : data.aws_instances.nlb_insts.ids[tid]]
}

where var.nlb_listeners is defined like this:

nlb_listeners = [
  {
    protocol    = "TCP"
    target_port = "80"
    health_port = "1936"
  },
  {
    protocol    = "TCP"
    target_port = "443"
    health_port = "1936"
  }
]

and var.elb_members.ids is like this:

"ids" = [
    "i-015604f88xxxxxx42",
    "i-0e4defceexxxxxxe5",
  ]

but I’m getting Invalid for_each argument error:

Error: Invalid for_each argument

on ../../modules/elb/balencer.tf line 46, in resource "aws_lb_target_group_attachment" "tgr_attachment": 46: for_each = { 47: for pair in setproduct(keys(aws_lb_target_group.nlb_target_groups), var.elb_members.ids) : "${pair[0]}:${pair[1]}" => { 48:
target_group = aws_lb_target_group.nlb_target_groups[pair[0]] 49:
instance_id = pair[1] 50: } 51: }

The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.

I cannot figure out why it’s either invalid or how this for_each cannot determine the values. Any idea what’s am I doing wrong here? Seriously got stuck in the middle and would really appreciate any help to put me to the right direction.

-S

=== Update: 02/23 ==========

@martin-atkins, I think I understand what you said but it seems give me the same error even for the instances that already exist. Anyway, this is my aws_instance resource:

resource "aws_instance" "inst" {
  count         = var.inst_count
  instance_type = var.inst_type
  depends_on    = [aws_subnet.snets]
  ami           = data.aws_ami.ubuntu.id

  # the VPC subnet
  subnet_id              = element(aws_subnet.snets.*.id, count.index)
  vpc_security_group_ids = [var.sg_default[var.idx], aws_security_group.secg.id]

  user_data = <<-EOF
          #!/bin/bash
          hostnamectl set-hostname ${var.vpc_names[var.idx]}${var.inst_role}0${count.index + 1}

          # Disable apt-daily.service & wait until `apt updated` has been killed
          systemctl stop apt-daily.service && systemctl kill --kill-who=all apt-daily.service
          while ! (systemctl list-units --all apt-daily.service | egrep -q '(dead|failed)')
          do sleep 1; done
  EOF

  # the public SSH key
  key_name = var.key_info

  tags = merge(
    var.common_tags,
    { "Name" = "${var.vpc_names[var.idx]}${var.inst_role}0${count.index + 1}" }
  )
}

Anything else you think can be done to get around that issue?

-S

Answer 1

Finally I've worked that out. Not sure if that's the best way of doing or not but now working for me error-free. Basically, I've change the data-lookup like this:

// List of instance attributes by role
data "aws_instance" "by_role" {
  for_each = {
    for ic in range(var.inst_count): "${var.inst_role}0${ic+1}" => ic
  }
  instance_tags  = {
    Name = "${var.vpc_names[var.idx]}${each.key}"
  }
  instance_id    = aws_instance.inst[substr(each.key,4,2)-1].id
}

and used that in the for_each like this:

for_each = {
    for pair in setproduct(keys(aws_lb_target_group.nlb_target_groups), keys(aws_instance.by_role)):
    "${pair[0]}:${pair[1]}" => {
      target_group = aws_lb_target_group.nlb_target_groups[pair[0]]
      member_name  = aws_instance.by_role[pair[1]]
    }
  }

Details are here, answered to my own question in the Terraform Community forum, just in case if someone else facing the same issue as mine.

-San

Answer 2

EC2 instance ids are not allocated until the instance has already been created, so the AWS provider cannot pre-calculate them during the plan phase. Because of that, they are not suitable for use as part of the key of a for_each map.

Instead, you'll need to use some other identifier for those instances that is determined by the configuration itself, rather than by data returned by the provider during apply. You didn't share the configuration for the instances themselves so I can't make a specific suggestion, but if they are all instances of the same resource created by count or for_each then a common choice is to use the index of each instance (for count) or the unique key of each instance (for for_each), so that elements of that map will be added and removed to correspond with the instances themselves being added and removed.