CODEWITHSUNDEEP
amazon-web-servicesamazon-ec2aws-application-load-balancerhttp-status-code-502
Manushi
Manushi

Reputation: 759

Does an Application Load Balancer does automatic health check on an unhealthy instance?

We have a private EC2 Linux instance running behind an ALB. There is only one instance running and no auto-scaling configured.

Sometimes ALB marks the instance as unhealthy for some reasons. This mostly happens when network traffic is high on the instance, which generally one or two hours. This behavior is unpredictable. So when try to access the web application which is deployed in the EC2 instance, we get 502 bad gateway. We reboot the EC2 instance and only then the issue is resolved.

Does an ALB perform a health check on a target group again after it marks it as unhealthy? Suppose an ALB marks the target group with one EC2 instance as unhealthy. ALB is configured to perform a health check every 30 seconds. Will it check for healthiness after 30 seconds after it marked as unhealthy on the same target group? Or will it look for new healthy instance?

I assume auto-scaling configuration may resolve this problem by setting AS group with 1 when an instance go unhealthy? Our AWS architect feels the Tomcat is creating memory leak when too many requests come at a time.Tomcat does not run in the EC2.

What is the way to troubleshoot this problem? I search for system logs and configured ALB access logs, but no clue is available.

In this link I see ALB routes requests to the unhealthy targets when no other healths target is available . https://docs.aws.amazon.com/elasticloadbalancing/latest/application/target-group-health-checks.html

My question is will ALB perform health check on the target group again after it marks it as unhealthy?

Upvotes: 1

Views: 1328

Answers (1)

LRutten
LRutten

Reputation: 1902

Indeed even when marked as unhealthy, the ALB continues the health checking. You can configure a 'healthy threshold count', which indicates how many 'healthy' responses should be received before an unhealthy host is marked as healthy again.

According to the docs:

When the health checks exceed HealthyThresholdCount consecutive successes, the load balancer puts the target back in service.

If your health check interval is 60 seconds, and the healthy threshold count is 3, it takes a minimum of 3 minutes before an unhealthy host will be marked healthy again.

Upvotes: 1

Related Questions