CODEWITHSUNDEEP
amazon-web-servicesaws-application-load-balancer
briancaffey
briancaffey

Reputation: 2558

Diagnosing slow response times for AWS Application Load Balancers

I am working on a POC for an application that uses CloudFront, ALB and Fargate. While I have been developing the application I noticed that the ALB sometimes has slow response times. I came across the following thread on the AWS forum. It is from 8 years ago, but I thought it might explain some of the behavior I am seeing:

https://forums.aws.amazon.com/thread.jspa?threadID=102879

The reason for the delay is because you have the ELB setup for multi-az without any application instances in the other 2 AZ's configured. Without instances in those AZ's requests will tend to fail because the ELb still returns IP addresses for those AZ's even if there are no active application instances. Please disable the other AZ's for now and continue your tests.

I have a similar setup. 2 AZs and only one Fargate task in my backend API Fargate service that the load balancer sends traffic to.

Here's what I get running dig against the ALB DNSName (ALB DNSName and IP addresses have been changed): 

dig my-alb-123.us-east-1.elb.amazonaws.com

; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> my-alb-123.us-east-1.elb.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52135
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;my-alb-123.us-east-1.elb.amazonaws.com. IN A

my-alb-123.us-east-1.elb.amazonaws.com. 60 IN A 51.4.143.123
my-alb-123.us-east-1.elb.amazonaws.com. 60 IN A 33.553.34.153

;; Query time: 3113 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Tue May 26
;; MSG SIZE  rcvd: 124

Running this again, I get the same result, but with a Query time of 0msec. Runnging this dozens of time, I see that most results return 0msec, and some that are ~3000msec.

For a staging environment, should I use one AZ?

Upvotes: 3

Views: 10954

Answers (1)

Marcin
Marcin

Reputation: 238587

Query time: 3113 msec

This is way too much. I checked on my 2-AZ ALB, and the times I am getting are about 60 msec, also in us-east-1:

dig xxxxx.us-east-1.elb.amazonaws.com

; <<>> DiG 9.16.2 <<>> xxxxx.us-east-1.elb.amazonaws.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56978
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;xxxx.elb.amazonaws.com.    IN A

;; ANSWER SECTION:
xxxxx.us-east-1.elb.amazonaws.com.  60 IN A 52.0.195.xx
xxxxx.us-east-1.elb.amazonaws.com.  60 IN A 50.19.163.xx

;; Query time: 53 msec
;; SERVER: 10.1.1.1#53(10.1.1.1)
;; WHEN: Wed May 27 16:08:53 AWST 2020
;; MSG SIZE  rcvd: 123

The times reported by dig are related to ALB's DNS resolution. And the two IPs shown are ALB nodes (one in each AZ). They do not depend on whether you have Fargate or EC2 instances.

For a staging environment, should I use one AZ?

If your Fargate service is set to only one AZ, any requests that will come to a node of ALB in a second zone, will have to be routed to the first zone anyway. There does not seem to be much benefit of having ALB in two AZs, but Fargate service limited to only one AZ.

Upvotes: 2

Related Questions