What is the best way to debug Terraform AWS Application Load Balancer validation error?

Question

I am trying to provision a demo web service on AWS with Terraform, and am encountering the following error.

Error: Error applying plan:

2 error(s) occurred:

* module.prod.module.web.module.web.aws_alb_listener.frontend: 1 error(s) occurred:

* aws_alb_listener.frontend: Error creating LB Listener: ValidationError: 'arn:aws:elasticloadbalancing:us-west-2:114416042199:loadbalancer/app/demo-svc-prod-alb/2a5f486a7b9d265a' is not a valid target group ARN
  status code: 400, request id: e3819755-799c-11e8-ac82-43dfdd4e44d1
* module.prod.module.web.module.web.aws_autoscaling_group.backend: 1 error(s) occurred:

* aws_autoscaling_group.backend: Error creating AutoScaling Group: ValidationError: Provided Load Balancers may not be valid. Please ensure they exist and try again.
  status code: 400, request id: e37efee9-799c-11e8-955a-c50a9e447dfa

What I don't understand is why the ARN is invalid, as it belongs to a resource created by Terraform. It seems perhaps suspicious that the ARN refers to elasticloadbalancing. Are there any gotchas to be aware of when working with an AWS application load balancer and an ASG? When using a classic ELB I didn't see this problem. Is there any way to get more useful information out of Terraform?

The relevant resources that throw the errors are:

resource "aws_alb_listener" "frontend" {
  load_balancer_arn       = "${aws_alb.frontend.arn}"
  port                    = "${local.https_port}"
  protocol                = "HTTPS"
  ssl_policy              = "ELBSecurityPolicy-TLS-1-2-2017-01"

  default_action {
    target_group_arn      = "${aws_alb.frontend.arn}"
    type                  = "forward"
  }
}

and

resource "aws_autoscaling_group" "backend" {
  name                    = "${local.cluster_name}-asg"
  launch_configuration    = "${aws_launch_configuration.backend.id}"
  availability_zones      = ["${data.aws_availability_zones.all.names}"]
  load_balancers          = ["${aws_alb.frontend.name}"]
  health_check_type       = "ELB"
  min_size                = "${var.min_size}"
  max_size                = "${var.max_size}"
  // This resource type uses different tags specification format.
  // A list comp over the locals tags map would sure come in handy to keep
  // things DRY.
  tags                    = [
    {
      key                 = "System"
      value               = "${var.tags["System"]}"
      propagate_at_launch = true
    },
    {
      key                 = "Environment"
      value               = "${local.tags["Environment"]}"
      propagate_at_launch = true
    },
    {
      key                 = "Owner"
      value               = "${local.tags["Owner"]}"
      propagate_at_launch = true
    },
    {
      key                 = "Description"
      value               = "${local.tags["Description"]}"
      propagate_at_launch = true
    }
  ]
}

The full code is available at https://github.com/mojochao/terraform-aws-web-stack/commit/a4bfe5d6362fddfb2934dc9a89344c304e59cef7.

Answer 1

You're referring to the wrong resources in both cases.

With the first error your listener is defined as:

resource "aws_alb_listener" "frontend" {
  load_balancer_arn       = "${aws_alb.frontend.arn}"
  port                    = "${local.https_port}"
  protocol                = "HTTPS"
  ssl_policy              = "ELBSecurityPolicy-TLS-1-2-2017-01"

  default_action {
    target_group_arn      = "${aws_alb.frontend.arn}"
    type                  = "forward"
  }
}

Note that the default_action takes a target_group_arn so you need to point it to your target group, not the load balancer itself.

So instead you should use:

resource "aws_alb_listener" "frontend" {
  load_balancer_arn       = "${aws_alb.frontend.arn}"
  port                    = "${local.https_port}"
  protocol                = "HTTPS"
  ssl_policy              = "ELBSecurityPolicy-TLS-1-2-2017-01"

  default_action {
    target_group_arn      = "${aws_alb_target_group.frontend.arn}"
    type                  = "forward"
  }
}

Because you only have a single listener rule defined you can also remove the aws_alb_listener_rule resource because it's doing the same thing as the default action on the listener anyway. You would only define rules separately if you wanted different traffic (either by host or by path) to go to different target groups.

Your second error comes from trying to attach the autoscaling group to an ELB classic by using the load_balancers parameter. As the aws_autoscaling_group resource docs mention you should use target_group_arns instead:

resource "aws_autoscaling_group" "backend" {
  name                    = "${local.cluster_name}-asg"
  launch_configuration    = "${aws_launch_configuration.backend.id}"
  availability_zones      = ["${data.aws_availability_zones.all.names}"]
  target_group_arns       = ["${aws_alb_target_group.frontend.arn}"]
  health_check_type       = "ELB"
  min_size                = "${var.min_size}"
  max_size                = "${var.max_size}"
  // This resource type uses different tags specification format.
  // A list comp over the locals tags map would sure come in handy to keep
  // things DRY.
  tags                    = [
    {
      key                 = "System"
      value               = "${var.tags["System"]}"
      propagate_at_launch = true
    },
    {
      key                 = "Environment"
      value               = "${local.tags["Environment"]}"
      propagate_at_launch = true
    },
    {
      key                 = "Owner"
      value               = "${local.tags["Owner"]}"
      propagate_at_launch = true
    },
    {
      key                 = "Description"
      value               = "${local.tags["Description"]}"
      propagate_at_launch = true
    }
  ]
}

This will automatically attach the autoscaling group to the ALB target group so you can also get rid of the aws_autoscaling_attachment resource which is doing the same thing. You would only normally use the aws_autoscaling_attachment resource if you defined your ALB target group and your autoscaling group separately and needed to link across them.