CODEWITHSUNDEEP
c#.nethttpwebrequest
Mark
Mark

Reputation: 981

Download file with timeout

I am attempting to just download the html from a webpage, but I want to give up after 10 seconds. The code below downloads the text just fine, but it can take longer than 10 seconds. I have the timeout set, but it is the StreamReading that takes a long time. What is the best way to stop any further processing after 10 seconds while still closing connections?

I am getting a WebException if req.GetResponse() takes longer than 10 seconds, but wr.GetResponseStream() reading is what is taking time. I also want to ensure that all connections are properly closed.

Code:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.Timeout = 10000;
req.ReadWriteTimeout = 10000;

using (WebResponse wr = req.GetResponse())
{
  Console.WriteLine("A: " + DateTime.Now.ToString(" HH:mm:ss:fff"));

  using (StreamReader sr = new StreamReader(wr.GetResponseStream(), true))
  {
     Console.WriteLine("B: " + DateTime.Now.ToString(" HH:mm:ss:fff"));
     var b = sr.ReadToEnd();
     Console.WriteLine("C: " + DateTime.Now.ToString(" HH:mm:ss:fff"));
  }
}

Sample Output:

A:  20:04:36:522
B:  20:04:36:522
C:  20:04:54:337

Elapsed Time: ~18 Seconds

Upvotes: 3

Views: 1134

Answers (1)

Motomotes
Motomotes

Reputation: 4237

The time is consumed in ReadToEnd

Use 

public virtual int Read(
    char[] buffer,
    int index,
    int count
)

set count to like 4000, even the slowest connections should have enough bandwidth to provide that 4000 characters or 8kB almost instantaneously.

Be sure to increment index of your buffer between each read, or you could use an 8kB buffer and just add the contents of it to a dynamic buffer each iteration.

Use Read inside a loop that checks the time and exits if greater than timeout or exits if Read returns a value less than count.

Also, you might want to look into async transfers, the right way to get data from the net: HttpWebRequest - Asynchronous Programming

Upvotes: 1

Related Questions