CODEWITHSUNDEEP
c#dnshttpclientsystem.net
tigrou
tigrou

Reputation: 4516

How to improve slow DNS with HttpClient?

I have one request for a specific domain that take a really long time to complete : 22 seconds in average. The request itself does not return a lot of data.

var httpClient = new HttpClient();  //instantiated at app start and reused 
var request = new HttpRequestMessage(HttpMethod.Get, "http://www.somedomain.com");
var result = await httpClient.SendAsync(request); //take a really long time

After some debugging, it seems related to DNS. If I run the exact same request but replace domain name by IP, it's fast (< 200 ms).

It's also fast if I run the request a second time using same HttpClient instance. However, if I run that second request after more than one minute it's slow again (probably because connections are closed automatically after 1 minute).

Is there a way to improve this ? (eg: to make sure resolved DNS entries would stay in cache for a long period). Also why is DNS so slow for that domain when using HttpClient ? If I try to access that domain using Chrome it's blazing fast (even after flushing DNS and clearing cache). A DNS lookup of that domain using an online service is also very fast. Reported TTL is 60 minutes.

EDIT: I have tried to increase ServicePointManager.DnsRefreshTimeout (which is set by default to 2 minutes). But it does not help.

Calling Dns.GetHostEntry("somedomain.com") is instantaneous and return the right IP. I don't know why request with HttpClient are so slow.

Upvotes: 3

Views: 4395

Answers (3)

56ka
56ka

Reputation: 1565

An other working solution using SocketsHttpHandler and HttpClient here : https://stackoverflow.com/a/70475741/1529139

It uses IPv4 over IPv6 at connection time.

It did the job for me

Upvotes: 0

tigrou
tigrou

Reputation: 4516

I found out what is wrong: the domain that is slow has an IPV6 address in the DNS records. This can be confirmed by calling this function:

Dns.GetHostEntry("somedomain.com")

First it tries to connect using IPV6. Eventually, a timeout occurs. Then it tries IPV4 (and it works). This explains why it is so slow. The solution I have found so far is to force IPV4 by resolving the domain name myself:

var uri = new Uri("somedomain.com");
var builder = new UriBuilder(uri);
var addresses = await Dns.GetHostAddressesAsync(uri.Host);    
builder.Host = addresses
                 .First(x => x.AddressFamily == AddressFamily.InterNetwork) //IPV4 only
                 .ToString(); 
var request = new HttpRequestMessage(HttpMethod.Get, builder.Uri);
request.Host = uri.Host;

Maybe there is a better way. I can't use SocketsHttpHandler since i'm stuck with .NET Framework 4.5

Upvotes: 3

Hrca12
Hrca12

Reputation: 31

It's not only about caching DNS query. If you create new client for every request, then before every request both TCP and HTTP handshakes are made -- which makes your single request slow. This is probably the reason why subsequent requests with "old" agent are faster -- handshakes are only made before first request, but not before second one.

Upvotes: -1

Related Questions