WebClient.DownloadString() is very slow downloading repeated

Question

This question has answered already many time, but unfortunately for me no answer is working.

I am simply using WebClient.DownloadString() and its execution is very slow.I have tried setting the WebClient.Proxy to null and GlobalProxySelection.GetEmptyWebProxy(), which didn't work and i am not anywhere near maxing my internet speed. What am I doing wrong?

public partial class WebForm1 : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        var res = GetData("tp");
        ShowData(res);
        res = GetData("nl");
        ShowData(res);
        res = GetData("sp");
        ShowData(res);
        res = GetData("en");
        ShowData(res);
        res = GetData("in");
        ShowData(res);
        res = GetData("st");
        ShowData(res);
        res = GetData("sh");
        ShowData(res);
        res = GetData("fl");
        ShowData(res);
        res = GetData("ce");
        ShowData(res);
    }

    public IEnumerable<object> GetData(string playListName)
    {
        try
        {
            String url = string.Empty;
            string jsonStr = string.Empty;

            if (playListName == "tp")
                url = "https://content.jwplatform.com/feeds/wDg7QXiZ.json";
            else if (playListName == "nl")
                url = "https://content.jwplatform.com/feeds/ZluqvHh7.json";
            else if (playListName == "sp")
                url = "https://content.jwplatform.com/feeds/CqjyIXVZ.json";
            else if (playListName == "en")
                url = "https://content.jwplatform.com/feeds/FEvaC7IT.json";
            else if (playListName == "in")
                url = "https://content.jwplatform.com/feeds/7HSiwnEm.json";
            else if (playListName == "st")
                url = "https://content.jwplatform.com/feeds/ioE6BWAD.json";
            else if (playListName == "sh")
                url = "https://content.jwplatform.com/feeds/XHZFlpw1.json";
            else if (playListName == "fl")
                url = "https://content.jwplatform.com/feeds/N2gtCgnE.json";
            else if (playListName == "ce")
                url = "https://content.jwplatform.com/feeds/Lw3otpHB.json";
            else
                url = string.Empty;

            using (var webClient = new WebClient())
            {
                webClient.Proxy = null; // not working
                //webClient.Proxy = GlobalProxySelection.GetEmptyWebProxy(); // not working
                jsonStr = webClient.DownloadString(url); // take 8 to 10 seconds
            }
            return jsonStr;
        }
        catch (Exception) { return null; }
    }
}

While call the url in browser-url is works fine

Answer 1

Hard to tell exactly what is the major reason for the slowness without delving into it (looking at a wider context of the code, monitoring the network and\or profiling the app).

Having said that, generally speaking there are a few things I can think of that may help:

1. If you can use HttpClient for multiple call then it might improve the performance. Unlike weblient to get the best performance you need to reuse the same instance of Httpclient (that might save you requests like dns look ups).

2. If you have many request running in parallel then you might want to know that .net allow you to run a maximum of two requests in parallel. You can change it by configuring the connection limit for your endpoint.

3. In case it's possible and you didn't configure this yet then you can set the httpclienthandler automaticDecompression property to except deflate or gzip content types. That will enable the server, in case it supports it, to compress the message before sending it back to your page.

Just keep in mind that the issue night be somewhere else - like a network problem or that an endpoint throttling.