CODEWITHSUNDEEP
c#
Jeremy Stein
Jeremy Stein

Reputation: 19661

How to view the headers sent by HttpWebRequest

I'm scraping a site with HttpWebRequest, but the site is returning an error. The page works fine when I hit it from my browser. I'd like to compare them to see what may be causing the error. I know how to intercept the request from my browser to inspect the headers, but how do I view the data sent by HttpWebRequest?

Upvotes: 7

Views: 18824

Answers (5)

user152771
user152771

Reputation: 39

You can get the headers from a HTTPWebRequest via the Headers property. From MSDN: http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.headers.aspx

Upvotes: 0

Chris Doggett
Chris Doggett

Reputation: 20757

The Net panel of Firebug will show all requests, including headers.

EDIT: Saw you already knew how to do it in a browser as soon as I posted. Try the Headers property:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.stackoverflow.com");
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Console.WriteLine(response.Headers);

Upvotes: 2

driis
driis

Reputation: 164291

In order to compare what you do in code, and what the browser does, I am sure a HTTP debugging tool such as Fiddler, would be the easiest solution.

Fiddler acts as a proxy between client and server, and displays all information sent over the HTTP protocol.

It is possible that you will need to configure your .NET app to use the proxy that Fiddler provides. This blog post provides details on the subject.

Upvotes: 9

Colin Fine
Colin Fine

Reputation: 3364

I don't know if there's a general solution. But if you're using Firefox, either of two add-ons will help: Firebug, or LiveHTTPHeaders.

Upvotes: -1

Carl Bergquist
Carl Bergquist

Reputation: 3942

http://www.fiddler2.com/fiddler2/ is a great tool for such things.

Upvotes: 7

Related Questions