Autoscaling group and ip target type for internal load balancers

Question

I'm trying to create an autoscaling group that is behind an internal network load balancer. The instances in the ASG group needs to talk to each other. From my tests and what I read, it means I have to register instances by IP address instead of instance ID.

I managed to get it working by adding the instances manually to the target group, but how am I supposed to do this automatically for instances that are managed by the autoscaling group? I'm using Terraform but I see no option to do that, neither can I find how to do it in the aws console. I don't think it's possible, given that when I try I get this error:

Provided Target Group 'arn:aws:elasticloadbalancing:us-east-2:xxxxxx:targetgroup/xxxx/xxxxxx' has invalid target type. Please ensure all provided Target Groups have target type of instance.

Any solution? Either how to register instances in an autoscaling group automatically by ip address or have an internal load balancer that allows instances to talk to each others with instance as target type. Thanks.

Answer 1

I had the same question - so sharing my findings in case it will help someone.

Target group has "Preserve client IP addresses" attribute which can be disabled and this will allow target EC2 to establish connection to other EC2s within the same ASG via Load Balancer. In that scenario you can still register targets by instance name instead of IP and benefit from automatic target's registration within target group.

Please refer to "Client IP preservation" section

https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#target-type

Please note it has limitations as described within provided link:

If you specify targets by instance ID, you might encounter TCP/IP connection limitations related to observed socket reuse on the targets. These connection limitations can occur when a client, or a NAT device in front of the client, uses the same source IP address and source port when connecting to multiple load balancer nodes simultaneously. If the load balancer routes these connections to the same target, the connections appear to the target as if they come from the same source socket, which results in connection errors. If this happens, the clients can retry (if the connection fails) or reconnect (if the connection is interrupted). You can reduce this type of connection error by increasing the number of source ephemeral ports or by increasing the number of targets for the load balancer. You can prevent this type of connection error by specifying targets by IP address or by disabling cross-zone load balancing.

Otherwise if you still would like to register targets by IP you can archive this using custom Lambda which will be triggered automatically by CloudWatch Auto Scaling event and based on event context register or deregister EC2 with target group.