Why isn't HttpClientHandler picking up and using default proxy settings correctly?

Question

We have a number of .NET Core 2.1 services, and some .NET Framework 4.7.1 applications running on a customer's Windows Server (2016 I think) box. The customer has a proxy set up (through Control Panel > Proxy Settings > LAN Settings) which all traffic needs to use. All of these services are making calls out to the cloud.

What we are seeing is some of the calls from the .NET Core services are reaching the cloud (such as logging, which we are able to view), but most other calls are timing out. We don't have any of these issues with the .NET Framework applications.

All of the services/applications are using HttpClient (single instance throughout each application) with HttpRequestMessage requests, rather than HttpWebRequest objects. I've read that if you do initialise the HttpClient with a HttpClientHandler and set the UseProxy property to true, and the Proxy property to null, it should use the default proxy settings set up in Windows, and I believe if you do not initialise the HttpClient with a HttpClientHandler, it will use a default one anyway which should have the same effect.

We've managed to track down the issue to how the HttpClient is using the default proxy settings, by making a test .Net Core console application and making calls to a number of cloud services. Interestingly, it seems the default proxy settings are used, because the first call works as expected. However, every call after that (to different cloud services) fails in exactly the same way as if it isn't using the proxy settings at all.

We've managed to solve this issue by actually providing the HttpClientHandler object passed in to the HttpClient with a WebProxy object configured to use the correct proxy. Once this is set, all the calls succeed. This is not the preferred solution though as it would involve the customer having to provide extra config for each service. They also don't necessarily know which machine these services will be running on, and may not know the proxy settings when configuring the services.

We also solved the issue another way, by disposing of the HttpClient after every single call and instantiating a new one. Obviously this can't be the solution though, as this is bad practice and could cause other issues (and would involve lots of code changes throughout multiple dependencies).

When we convert the .NET Core 2.1 console application to a .NET Framework 4.7.1 application we also don't see the issue.

We also don't see any issue if we use HttpWebRequest objects to make the requests.

This is the current, failing console application code. We are using async correctly in our services rather than calling Result. This doesn't seem to make a difference.

var httpClient = new HttpClient();

Console.WriteLine("Hitting service 1 API");

var request = new HttpRequestMessage(HttpMethod.Get, $"{baseUrl}/service1/serviceinfo");

var result = httpClient.SendAsync(request).Result;

Console.WriteLine($"Response code: {result.StatusCode}");           

var responseMessage = result.Content.ReadAsStringAsync().Result;
Console.WriteLine(responseMessage);

Thread.Sleep(2000);

Console.WriteLine("Hitting service 2 API");

request = new HttpRequestMessage(HttpMethod.Get, $"{baseUrl}/service2/serviceinfo");

result = httpClient.SendAsync(request).Result;

Console.WriteLine($"Response code: {result.StatusCode}");          

responseMessage = result.Content.ReadAsStringAsync().Result;
Console.WriteLine(responseMessage);            

Thread.Sleep(2000);

Console.WriteLine("Hitting service 3 API");

request = new HttpRequestMessage(HttpMethod.Get, $"{baseUrl}/service3/serviceinfo");

result = httpClient.SendAsync(request).Result;

Console.WriteLine($"Response code: {result.StatusCode}");            

responseMessage = result.Content.ReadAsStringAsync().Result;
Console.WriteLine(responseMessage);

Expected Result: All three calls should go through and receive a 200 response back, whether you have a proxy configured or not.

Actual Result: When not using a proxy, we get the expected result, though when using a proxy the first calls succeeds as expected, but the next fails with a SocketException on our local test machine, or a GatewayTimeout on the customer machine (I believe this is just the way in which we're blocking all traffic that isn't going through the proxy).

Expected result occurs if a new HttpClient is used for each call.

Expected result occurs if a HttpWebRequest is used instead of using the HttpClient

Expected result occurs if HttpClient is initialised with a HttpClientHandler that has had the proxy settings explicitly set.

Answer 1

After some more digging around, we have found the following solution:

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

Apparently in .NET Core 2.1, the HttpClient is forced to use a new HttpClientHandler. You can either use the AppContext switch above, or initialise your HttpClient with the old handler. This seems like a clear bug/regression with .NET Core, but I'm aware I can't seem to find anybody else online that has had this exact issue. Anymore suggestions are welcome. Maybe a bug needs reporting?