CODEWITHSUNDEEP
c#console-application
Matt McManis
Matt McManis

Reputation: 4675

HttpWebRequest StreamReader Progress Percent

I'm using C# Console Application.

I'm parsing an HTML page with HttpWebRequest and StreamReader.

How can I add a Console.Write to the StreamReader that tracks progress %?

HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(url);

req.UserAgent = "MOZILLA/5.0 (WINDOWS NT 6.1; WOW64) APPLEWEBKIT/537.1 (KHTML, LIKE GECKO) CHROME/21.0.1180.75 SAFARI/537.1";
req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";

HttpWebResponse response = (HttpWebResponse)req.GetResponse();

string page = string.Empty;

using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
    while (sr.Peek() >= 0)
    {
        page = sr.ReadToEnd();
    }
}

Similar to DownloadProgressChanged with WebClient for downloading files, but not web pages.

WebClient wc = new WebClient();
wc.DownloadProgressChanged += new DownloadProgressChangedEventHandler(wc_DownloadProgressChanged);
wc.DownloadFileAsync(downloadUrl, path);

public static void wc_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
    double bytesIn = double.Parse(e.BytesReceived.ToString());
    double totalBytes = double.Parse(e.TotalBytesToReceive.ToString());
    double percentage = bytesIn / totalBytes * 100;

    Console.Write("\rProgress: {0}%", percentage);
}

Upvotes: 0

Views: 195

Answers (1)

Gabriel Luci
Gabriel Luci

Reputation: 40958

I've never done this, but you could use the HttpWebResponse.ContentLength property (which is the value from the Content-Length HTTP header) to get the total number of bytes that you can expect. Then you could use StreamReader.Read() to read a little bit at a time until you reach the the size that you got from ContentLength.

Something like this (totally untested):

HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://");

req.UserAgent = "MOZILLA/5.0 (WINDOWS NT 6.1; WOW64) APPLEWEBKIT/537.1 (KHTML, LIKE GECKO) CHROME/21.0.1180.75 SAFARI/537.1";
req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";

HttpWebResponse response = (HttpWebResponse)req.GetResponse();

var page = new StringBuilder();

using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
    while (!sr.EndOfStream)
    {
        var buffer = new char[4096];
        var numRead = sr.Read(buffer, 0, buffer.Length);
        html.Append(buffer, 0, numRead);

        var percentage = Convert.ToInt32(Convert.ToDouble(html.Length) / Convert.ToDouble(response.ContentLength) * 100);

        Console.Write("\rProgress: {0}%", percentage);
    }
}
Console.Write("\rProgress: 100%");

Now I know that the Content-Length header is the number of bytes in the response, and I'm using StringBuilder.Length to monitor progress, which is the number of characters. In Unicode, a single character can equal more than one bytes, so it's not exactly right to calculate it this way, but it's close enough.

Upvotes: 1

Related Questions