CODEWITHSUNDEEP
c#httpwebrequestbing
Jarrod Maxwell
Jarrod Maxwell

Reputation: 11

HttpWebRequest has empty response requesting a search from Bing

I have the following code that sends a HttpWebRequest to Bing. When I request the url below though it returns what appears to be an empty response when it should be returning a list of results.

var response = string.Empty;
var httpWebRequest = WebRequest.Create("http://www.bing.com/search?q=stackoverflow&count=100") as HttpWebRequest;

httpWebRequest.Method = WebRequestMethods.Http.Get;
httpWebRequest.Headers.Add("Accept-Language", "en-US");
httpWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Win32)";
httpWebRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");

using (var httpWebResponse = httpWebRequest.GetResponse() as HttpWebResponse)
{
    Stream stream = null;
    using (stream = httpWebResponse.GetResponseStream())
    {
        if (httpWebResponse.ContentEncoding.ToLower().Contains("gzip"))
            stream = new GZipStream(stream, CompressionMode.Decompress);
        else if (httpWebResponse.ContentEncoding.ToLower().Contains("deflate"))
            stream = new DeflateStream(stream, CompressionMode.Decompress);

        var streamReader = new StreamReader(stream, Encoding.UTF8);
        response = streamReader.ReadToEnd();
    }
}

Its pretty standard code for requesting and receiving a web page. Any ideas why the response is empty? Thanks in advance.

EDIT I left off a query string parameter in the url. I also had &count=100 which I have now corrected. It seems to work for values of 50 and below but returns nothing when larger. This works ok when in the browser, but not for this web request.

It makes me think the issue is that the response is large and HttpWebResponse is not handling that for me the way I have it set up. Just a guess though.

Upvotes: 1

Views: 3372

Answers (2)

AnthonyWJones
AnthonyWJones

Reputation: 189505

Your code works fine on my machine.

I suggest you get yourself a copy of Fiddler and examine the actual HTTP sesssion occuring. May be a proxy or firewall thing.

Upvotes: 1

Jan Jongboom
Jan Jongboom

Reputation: 27342

This works just fine on my machine. Perhaps you are IP banned from Bing?

Upvotes: 1

Related Questions