Why i'm getting time out exception and how should i handle this when using httpwebrequest to download a file?

Question

This is the code:

HttpWebRequest request;
        int currentIndex = 0;
        void fileDownloadRadar(string uri, string fileName)
        {
            if (splash != null)
            {
                if (!splash.IsDisposed)
                    splash.UpdateProgressBar(0);
            }
            /*_request = WebRequest.Create(uri) as HttpWebRequest;
            _request.CookieContainer = new CookieContainer();
            _request.AllowAutoRedirect = true;
            _responseAsyncResult = _request.BeginGetResponse(ResponseCallback, null);*/
            request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(uri);
            request.CookieContainer = new CookieContainer();
            request.AllowAutoRedirect = true;

            HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            long contentLength = response.ContentLength;
            if (response.ContentType == "")
            {
                Logger.Write("ContentType is Empty download was not fine !!!!!");
            }
            if ((response.StatusCode == HttpStatusCode.OK ||
                response.StatusCode == HttpStatusCode.Moved ||
                response.StatusCode == HttpStatusCode.Redirect) &&
                response.ContentType.StartsWith("image", StringComparison.OrdinalIgnoreCase))
            {
                Logger.Write("ContentType is not empty meaning download is fine");
                using (Stream inputStream = response.GetResponseStream())
                using (Stream outputStream = File.OpenWrite(fileName))
                {
                    byte[] buffer = new byte[4096];
                    int bytesRead;
                    do
                    {
                        bytesRead = inputStream.Read(buffer, 0, buffer.Length);
                        currentIndex += bytesRead;
                        double percentage = (double)currentIndex / contentLength;
                        if (splash != null)
                        {
                            if (!splash.IsDisposed)
                                splash.UpdateProgressBar((int)(percentage * 100));
                        }
                        outputStream.Write(buffer, 0, bytesRead);
                    } while (bytesRead != 0);
                    if (splash != null)
                    {
                        if (!splash.IsDisposed)
                        {
                            splash.UpdateProgressBar(100);
                        }
                    }
                }

            }
            else
            {
                timer1.Stop();
                timer3.Start();
            }
            if (splash == null)
                FinishWebRequest();
        }

The exception is on the line:

bytesRead = inputStream.Read(buffer, 0, buffer.Length);

The exception is: WebException

The operation has timed out

I see that when the exception happened the variable bytesRead value is 1360 And buffer values is 4096

System.Net.WebException occurred
  HResult=-2146233079
  Message=The operation has timed out.
  Source=System
  StackTrace:
       at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
       at mws.Form1.fileDownloadRadar(String uri, String fileName) in d:\C-Sharp\Download File\Downloading-File-Project-Version-012\Downloading File\Form1.cs:line 914
  InnerException: 

Line 914 is:

bytesRead = inputStream.Read(buffer, 0, buffer.Length);

Maybe i should put try and catch somewhere ?

My code is downloading a file then a timer is running when the timer count 5 minutes it's downloading the same file again and so on over again nonstop every 5 minutes i'm calling this method to download the file.

EDIT this is my code after changed it according to the answers:

HttpWebRequest request;
        int currentIndex = 0;
        void fileDownloadRadar(string uri, string fileName)
        {
            if (splash != null)
            {
                if (!splash.IsDisposed)
                    splash.UpdateProgressBar(0);
            }
            try
            {
                request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(uri);
                request.Timeout = 10000;
                request.CookieContainer = new CookieContainer();
                request.AllowAutoRedirect = true;
                using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
                {
                    long contentLength = response.ContentLength;
                    if (response.ContentType == "")
                    {
                        Logger.Write("ContentType is Empty download was not fine !!!!!");
                    }
                    if ((response.StatusCode == HttpStatusCode.OK ||
                        response.StatusCode == HttpStatusCode.Moved ||
                        response.StatusCode == HttpStatusCode.Redirect) &&
                        response.ContentType.StartsWith("image", StringComparison.OrdinalIgnoreCase))
                    {
                        Logger.Write("ContentType is not empty meaning download is fine");
                        using (Stream inputStream = response.GetResponseStream())
                        using (Stream outputStream = File.OpenWrite(fileName))
                        {
                            byte[] buffer = new byte[4096];
                            int bytesRead;
                            do
                            {
                                bytesRead = inputStream.Read(buffer, 0, buffer.Length);
                                currentIndex += bytesRead;
                                double percentage = (double)currentIndex / contentLength;
                                if (splash != null)
                                {
                                    if (!splash.IsDisposed)
                                        splash.UpdateProgressBar((int)(percentage * 100));
                                }
                                outputStream.Write(buffer, 0, bytesRead);
                            } while (bytesRead != 0);
                            if (splash != null)
                            {
                                if (!splash.IsDisposed)
                                {
                                    splash.UpdateProgressBar(100);
                                }
                            }
                        }

                    }
                    else
                    {
                        timer1.Stop();
                        timer3.Start();
                    }
                    if (splash == null)
                        FinishWebRequest();
                }
            }
            catch (WebException ex)
            {
                if (ex.Status == WebExceptionStatus.Timeout)
                {
                    Logger.Write(ex.Status.ToString());
                }
            }
        }

But after it was working fine for maybe an hour or so and downloaded the file few times now i got the exception time out again:

The operation has timed out

This time on the line:

using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())

And the exception message:

System.Net.WebException occurred
  HResult=-2146233079
  Message=The operation has timed out
  Source=System
  StackTrace:
       at System.Net.HttpWebRequest.GetResponse()
       at mws.Form1.fileDownloadRadar(String uri, String fileName) in d:\C-Sharp\Download File\Downloading-File-Project-Version-012\Downloading File\Form1.cs:line 899
  InnerException:

I changed the response and using the using Also added: request.Timeout = 10000; And added also try and catch.

Answer 1

The first step is to put your code inside try..catch

   try
   {
     //code 
   }
   catch (WebException ex)
   {
       if (ex.Status == WebExceptionStatus.Timeout)
       {
           //log your exception
       }
   }

Then wrap the web response with a using block because it automatically call close and also dispose the object

  using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
  {
      //your code
  }

Lastly, I see in your code you have attempted to use asynchronous request, maybe you can consider that if you're dealing with huge files to deal with a timeout issue. Hope it helps.

Answer 2

The timeouts you are seeing are caused by ServicePointManager - class that manages the collection of ServicePoint objects which provides connection management for HTTP connections.

In short, whenever you create a request and the response, the important thing is to call the Close() method on your response object. If not, it will prevent any of the connections in the ServicePoint object from being freed, so at some point, after you have made the critical number of request, no additional requests can be served and you will experience the timeouts.

For more in depth information check out this article: http://blogs.msdn.com/b/jpsanders/archive/2009/05/20/understanding-maxservicepointidletime-and-defaultconnectionlimit.aspx

HttpWebRequest req = CreateRequest();
HttpWebResponse res = req.GetResponse() as HttpWebResponse;
...
res.Close();  //you should always close the response

EDIT:

as @Kevin said in the comment, another solution would be to wrap the response in a using statement and that would automatically close the connection:

using(HttpWebResponse res = req.GetResponse() as HttpWebResponse)
{
    ...
}

Answer 3

System.Net.WebException: The operation has timed out

This may fit for your problem, as it is regarded to a TimeOut Exception. The most helpful answer relates to:

https://msdn.microsoft.com/en-us/library/system.net.webrequest.timeout.aspx