Reputation: 873

How to handle inconsistent HttpRequestExceptions when calling external API from web application

While trying to call an external API from react/.NET applications using HttpClient in the background, sometimes we are seeing http exceptions like this:

System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. (--url--)

System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (--url--)

System.Net.Sockets.SocketException (10061): No connection could be made because the target machine actively refused it.

This is the code to create the HttpClient:

services.AddHttpClient("SearchClient", client =>
{
    client.BaseAddress = new Uri(settings.SearchUrl);
    client.DefaultRequestHeaders.Add("Accept", "application/json");
    client.Timeout = TimeSpan.FromSeconds(settings.APITimeoutInSeconds);
});

services.AddHttpClient("LoginClient", client =>
{
    client.BaseAddress = new Uri(settings.LoginUrl);
    client.DefaultRequestHeaders.Add("Accept", "application/json");
    client.Timeout = TimeSpan.FromSeconds(settings.APITimeoutInSeconds);
});

Will instantiate those instances using below code inside singleton scoped instance class,

var client = _httpClientFactory.CreateClient("LoginClient");

using (var response = await client.PostAsync(requestUri, content))
{
    if (response.IsSuccessStatusCode)
    {
        var responseContent = await response.Content.ReadAsStringAsync();
        res = JsonConvert.DeserializeObject<TokenResponseModel>(responseContent);
    }
    else
    {
        apiLog.Response = response;
    }
}

Any idea how to fix these problems?

Upvotes: 1

Answers (1)

Teodor Mihail

Reputation: 906

Topology

HTTP/HTTPS is a networking protocol that is built over the TCP/IP networking protocol. The TCP/IP protocol uses the SYN, ACK, SYN/ACK paradigm. This paradigm further translates into the fact that a packet transmission session will follow the following steps: The sender sends a package to the receiver, the receiver receives the package and checks the checksum within the TCP header of the package, then the receiver responds to the sender that either the package was received successfully, either that the package was received but has corrupt data, or either the receiver does not respond to the sender at all if the package was not received. This paradigm was put in place for the TCP/IP protocol due to the fact that its purpose is to ensure the integrity of the data transmitted using this protocol. This means that the behavior that you are experiencing is normal because many variables can intervene during the data transmission process.

Solution

Increase the connection timeout

// INCREASE THE DURATION OF THE DATA TRANSMISSION SESSION

var client = _httpClientFactory.CreateClient("LoginClient");
client.Timeout = TimeSpan.FromSeconds(5);

using (var response = await client.PostAsync(requestUri, content))
{
    if (response.IsSuccessStatusCode)
    {
        var responseContent = await response.Content.ReadAsStringAsync();
        res = JsonConvert.DeserializeObject<TokenResponseModel>(responseContent);
    }
    else
    {
        apiLog.Response = response;
    }
}

Due to the fact that many factors can intervene within the data transmission process the connection timeout can be increased accordingly with the specificities of the current situation in order to increase the rate of success regarding the data transmission operation.

Calculate the connection speed and set the timeout

        // PAYLOAD TO SEND TO THE RECEIVER
        byte[] payload_to_send = new byte[2000000];

        // CREATE THE HTTP CLIENT OBJECT
        HttpClient client = new HttpClient();

        // SET THE BASE ADDRESS OF THE HTTP CLIENT OBJECT
        client.BaseAddress = new Uri(NavigationManager.BaseUri);

        // CREATE A TEST PAYLOAD
        byte[] test_payload = new byte[204800];

        // SET THE CONTENT TO BE SENT AS A TEST PAYLOAD AS A BYTE ARRAY
        ByteArrayContent http_payload = new ByteArrayContent(test_payload);

        // CREATE A STOPWATCH OBJECT TO MONITOR THE TIME TOOK BY EACH DATA TRANSFER PROCESS
        System.Diagnostics.Stopwatch latency_time_counter = new System.Diagnostics.Stopwatch();

        // INTEGER THAT STORES HOW MANY BYTES PER SECOND TOOK TO TRANSFER THE TEST PAYLOAD
        long download_bytes_per_second = 0;

        // TRANSFER THE PAYLOAD 10 TIMES
        for (int i = 0; i < 10; i++)
        {
            latency_time_counter.Start();
            await client.PostAsync("/speed-test-api/auth/get-account", http_payload);
            latency_time_counter.Stop();

            // INCREMENT THE 'download_bytes_per_second' BY THE NUMBER OF BYTES PER SECOND TOOK TO TRANSFER THE PAYLOAD FOR EACH ITERATION
            download_bytes_per_second += (long)(test_payload.Length * (1000 / latency_time_counter.Elapsed.TotalMilliseconds));
            Console.WriteLine("Bytes per second: " + download_bytes_per_second);

            latency_time_counter.Reset();
        }

        // DIVIDE THE TOTAL NUMBER OF BYTES PER SECOND BY THE NUMBER OF ITERATIONS TO GET THE AVERAGE BYTES PER SECOND
        download_bytes_per_second = download_bytes_per_second / 10;

        // ESTIMATION ERROR IN SECONDS
        int timeout_rounding_error = 1;

        // CALCULATE THE OPTIMAL TIMEOUT BASED ON THE LENGTH OF THE PAYLOAD TO BE SENT
        long download_optimal_connection_timeout = (payload_to_send.Length / download_bytes_per_second) + timeout_rounding_error;

        Console.WriteLine("\n\nAverage bytes per second: " + download_optimal_connection_timeout);

        // SET THE CONNECTION TIMEOUT
        client = new HttpClient();
        client.Timeout = TimeSpan.FromSeconds(download_optimal_connection_timeout);

In order to calculate the optimal data transmission timeout, the data transfer speed must be calculated and the timeout must be set accordingly. An HttpClient object is created and a loop is sending a 20 Kilobytes payload to the receiver, while measuring the number of milliseconds took to transfer the payload each iteration. This is calculated by multiplying the number of bytes in the payload by 1000 milliseconds and dividing it by the total number of milliseconds took to send the payload. Because the number of bytes per second is the targeted value to be extracted, 1000 milliseconds are divided by the total number of miliseconds to get the ratio for each second, for example if the time took to transfer 1000 bytes is 10000 milliseconds (10 seconds ), then the number of bytes per second is 100, because 10000 milliseconds is 10 times bigger than 1000 milliseconds, so the result will be 0.1 ( signifying that 1000 milliseconds are 10% of the total time took ) and multiplying the total number of bytes within the payload by 0.1 will result in 10% of the total payload being transferred each second, thus resulting in the following formula: BPS = TPB * 1000 / TT, where BPS = Bytes per second, TPB = Total payload bytes, and TT = Total time. Every iteration the variable that holds the speed in bytes per second is incremented by TPB. After the iteration finishes the variable that holds the speed in bytes per second is divided by 10 to get the average bytes per second speed. Afterwards the length of the payload to be sent is divided by the bytes per second speed to get in how many seconds the data to be sent will be transferred and a timeout rounding error is added for this number, which in this case is one second. You can find the mathematical principles and logic behind this at this link: https://www.baeldung.com/cs/calculate-internet-speed-ping

Error correction mechanism

        // SET THE CONNECTION TIMEOUT
        client = new HttpClient();
        client.BaseAddress = new Uri(NavigationManager.BaseUri);
        client.Timeout = TimeSpan.FromSeconds(download_optimal_connection_timeout);

        // SET THE MAXIMUM NUMBER OF RETRIES
        int max_retries = 10;
        var responseContent = new object();

        // ITERATE UNTIL THE EITHER THE TRANSFER IS SUCCESSFUL OR EITHER IT FAILS
        for(int i = 0; i < max_retries; i++)
        {
            try
            {
                using (var response = await client.PostAsync("/login/insert-account", new StringContent(Encoding.UTF8.GetString(payload_to_send))))
                {
                    // IF THE PAYLOAD TRANSFER IS SUCCESSFUL READ THE CONTENT AND BREAK THE LOOP
                    if (response.IsSuccessStatusCode)
                    {
                        responseContent = await response.Content.ReadAsStringAsync();
                        break;
                    }
                    else
                    {
                        // REPLACE WITH OWN LOGGING MECHANISM
                        Console.WriteLine(response);
                    }
                }
            }
            catch(Exception E)
            {
                // REPLACE WITH OWN LOGGING MECHANISM
                Console.WriteLine(E.Message);
            }
        }

Due to the fact that many variables can intervene into the payload transfer process, a payload transfer mechanism must be implemented. The error correction mechanism will retry to send the specified payload a certain number of times until the payload is either sent or either until the maximum number of retries is achieved.

Error correction mechanism and connection timeout auto-configuration based on the connection speed

        Console.WriteLine("\n\n");


        // PAYLOAD TO SEND TO THE RECEIVER
        byte[] payload_to_send = new byte[2000000];

        // CREATE THE HTTP CLIENT OBJECT
        HttpClient client = new HttpClient();

        // SET THE BASE ADDRESS OF THE HTTP CLIENT OBJECT
        client.BaseAddress = new Uri(NavigationManager.BaseUri);

        // CREATE A TEST PAYLOAD
        byte[] test_payload = new byte[204800];

        // SET THE CONTENT TO BE SENT AS A TEST PAYLOAD AS A BYTE ARRAY
        ByteArrayContent http_payload = new ByteArrayContent(test_payload);

        // CREATE A STOPWATCH OBJECT TO MONITOR THE TIME TOOK BY EACH DATA TRANSFER PROCESS
        System.Diagnostics.Stopwatch latency_time_counter = new System.Diagnostics.Stopwatch();

        // INTEGER THAT STORES HOW MANY BYTES PER SECOND TOOK TO TRANSFER THE TEST PAYLOAD
        long download_bytes_per_second = 0;

        // TRANSFER THE PAYLOAD 10 TIMES
        for (int i = 0; i < 10; i++)
        {
            latency_time_counter.Start();
            await client.PostAsync("/speed-test-api/", http_payload);
            latency_time_counter.Stop();

            // INCREMENT THE 'download_bytes_per_second' BY THE NUMBER OF BYTES PER SECOND TOOK TO TRANSFER THE PAYLOAD FOR EACH ITERATION
            download_bytes_per_second += (long)(test_payload.Length * (1000 / latency_time_counter.Elapsed.TotalMilliseconds));
            Console.WriteLine("Bytes per second: " + download_bytes_per_second);

            latency_time_counter.Reset();
        }

        // DIVIDE THE TOTAL NUMBER OF BYTES PER SECOND BY THE NUMBER OF ITERATIONS TO GET THE AVERAGE BYTES PER SECOND
        download_bytes_per_second = download_bytes_per_second / 10;

        // ESTIMATION ERROR IN SECONDS
        int timeout_rounding_error = 1;

        Console.WriteLine("\n\nAverage bytes per second: " + download_bytes_per_second);

        // CALCULATE THE OPTIMAL TIMEOUT BASED ON THE LENGTH OF THE PAYLOAD TO BE SENT
        long download_optimal_connection_timeout = (payload_to_send.Length / download_bytes_per_second) + timeout_rounding_error;

        Console.WriteLine("\n\nTime to transfer the payload: " + download_optimal_connection_timeout);

        // SET THE CONNECTION TIMEOUT
        client = new HttpClient();
        client.BaseAddress = new Uri(NavigationManager.BaseUri);
        client.Timeout = TimeSpan.FromSeconds(download_optimal_connection_timeout);

        // SET THE MAXIMUM NUMBER OF RETRIES
        int max_retries = 10;
        var responseContent = new object();

        // ITERATE UNTIL THE EITHER THE TRANSFER IS SUCCESSFUL OR EITHER IT FAILS
        for(int i = 0; i < max_retries; i++)
        {
            try
            {
                using (var response = await client.PostAsync("/login/insert-account", new StringContent(Encoding.UTF8.GetString(payload_to_send))))
                {
                    // IF THE PAYLOAD TRANSFER IS SUCCESSFUL READ THE CONTENT AND BREAK THE LOOP
                    if (response.IsSuccessStatusCode)
                    {
                        responseContent = await response.Content.ReadAsStringAsync();
                        break;
                    }
                    else
                    {
                        // REPLACE WITH OWN LOGGING MECHANISM
                        Console.WriteLine(response);
                    }
                }
            }
            catch(Exception E)
            {
                // REPLACE WITH OWN LOGGING MECHANISM
                Console.WriteLine(E.Message);
            }
        }

To maximize the chances that the payload to be sent will be sent successfully, the timeout of the connection must be set accordingly with the connection speed and an error correction mechanism must be implemented in order to ensure that the payload to be sent has the highest chance of being sent successfuly.