CODEWITHSUNDEEP
c#
Amged Rustom
Amged Rustom

Reputation: 1868

Get webpage page content and HTTP status code in C#

In a C# Windows Forms application I can get the contents of a webpage using:

string content = webClient.DownloadString(url);

And I can get the HTTP header using:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
string response = ((HttpWebResponse)request.GetResponse()).StatusCode.ToString();

Is there a way to get both the contents and the HTTP status code (if it fails) in one trip to the server instead of twice?

Thanks.

Upvotes: 6

Views: 13331

Answers (4)

CodeCaster
CodeCaster

Reputation: 151594

WebClient

I assume you use WebClient because its easy webrequest-to-string handling. Unfortunately, WebClient does not expose the HTTP response code. You can either assume the response was positive (2xx) unless you get an exception and read it:

try
{
    string content = webClient.DownloadString(url);
}
catch (WebException e)
{
    HttpWebResponse response = (System.Net.HttpWebResponse)we.Response;     
    var statusCode = response.StatusCode;
}

Or if you're really interested in the success code you can use reflection as explained here.

HttpClient

You can also use HttpClient if you're on .NET 4.5, which does expose the response code, as explained here:

using (HttpClient client = new HttpClient())
{
    HttpResponseMessage response = await client.GetAsync(url);

    string content = await response.Content.ReadAsStringAsync();
    var statusCode = response.StatusCode;       
}

HttpWebRequest

Alternatively, you can just use HttpWebRequest to get the status and response as explained here:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
var response = (HttpWebResponse)request.GetResponse();

using (Stream stream = response.GetResponseStream())
{
   StreamReader reader = new StreamReader(stream);

   string content = reader.ReadToEnd();
   var statusCode = response.StatusCode;    
}

Upvotes: 6

Miguel Angelo
Miguel Angelo

Reputation: 24192

You can read the data from the Stream inside the HttpWebResponse object:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
    HttpStatusCode statusCode = ((HttpWebResponse)response).StatusCode;
    string contents = reader.ReadToEnd();
}

In this way you will have to detect the encoding by hand, or using a library to detect encoding. You can read the encoding as a string from the HttpWebResponse object as well, when one exists, it is inside the ContentType property. If the page is Html, then you will have to parse it for a possible encoding change in the top of the document or inside the head.

Read handling the encoding from ContentType header

var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
string content;
HttpStatusCode statusCode;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
{
    var contentType = response.ContentType;
    Encoding encoding = null;
    if (contentType != null)
    {
        var match = Regex.Match(contentType, @"(?<=charset\=).*");
        if (match.Success)
            encoding = Encoding.GetEncoding(match.ToString());
    }

    encoding = encoding ?? Encoding.UTF8;

    statusCode = ((HttpWebResponse)response).StatusCode;
    using (var reader = new StreamReader(stream, encoding))
        content = reader.ReadToEnd();
}

Upvotes: 8

KoViMa
KoViMa

Reputation: 382

And I can get the HTTP header using: request.Method = "GET";

Method GET returns HEAD and BODY sections in response. HTTP also support a method HEAD - which returns HEAD section only.

You can get BODY from HttpWebResponse using GetResponseStream method.

Upvotes: 0

major-mann
major-mann

Reputation: 2652

I think, you have not realised, that in the second case you have access to the content as well (although it takes a little more effort to get as a string).

Look at the Microsoft documentation: http://msdn.microsoft.com/en-us/library/system.net.httpwebresponse.getresponsestream(v=vs.110).aspx which shows you how to ge a response stream from the web response, and then how to get the string data from that stream.

Upvotes: 0

Related Questions