CODEWITHSUNDEEP
c#.nethttpwebrequest
Tim Martin
Tim Martin

Reputation: 3697

HttpWebRequest.GetResponse() hangs the second time it is called

I'm trying to fetch a series of files via HTTP, using HttpWebRequest. The first request goes through fine, but the second time through the same code GetResponse() hangs and times out. WireShark shows that no HTTP traffic is being sent for the second request, so it would appear that it's an API problem.

After some investigation, I've discovered that it has to do with specifying the content-length: if I leave this out, then the code works fine.

My code is:

HttpWebRequest  httpWebRequest = ConfigureRequest();

using (WebResponse webResponse = httpWebRequest.GetResponse())
    // On the second iteration we never get beyond this line
{
    HttpWebResponse httpWebResponse = webResponse as HttpWebResponse;

    using (Stream webResponseStream = httpWebResponse.GetResponseStream())
    {
        if (webResponseStream != null)
        {
            // Read the stream
        }
    }

    statusCode = httpWebResponse.StatusCode;
    httpWebResponse.Close();
}

The symptoms seem very similar to this question and this question, but in both cases the advice given is to dispose of the WebResponse, which I'm already doing.

Edit In response to Gregory, here is ConfigureRequest():

private HttpWebRequest ConfigureRequest()
{
    string          sUrl            = CreateURL(bucket, key);
    HttpWebRequest  httpWebRequest  = WebRequest.Create(sUrl) as HttpWebRequest;

    httpWebRequest.AllowWriteStreamBuffering = false;
    httpWebRequest.AllowAutoRedirect = true;
    httpWebRequest.UserAgent = this.m_sUserAgent;
    httpWebRequest.Method = "GET";
    httpWebRequest.Timeout = this.m_iTimeout;

    // *** NB: This line was left out of my original posting, and turned out to be
    // crucial
    if (m_contentLength > 0)
        httpWebRequest.ContentLength = m_contentLength;

    httpWebRequest.Headers.Add(StaticValues.Amazon_AlternativeDateHeader, timestamp);
    httpWebRequest.Headers.Add(StaticValues.HttpRequestHeader_Authorization, StaticValues.Amazon_AWS + " " + aWSAccessKeyId + ":" + signature);

    return httpWebRequest;
}

Edit: It turns out I committed the cardinal sin of removing code from my question that I hadn't verified was unrelated to the problem. I had removed the following lines:

    if (m_contentLength > 0)
        httpWebRequest.ContentLength = m_contentLength;

because I thought that content-length would never be specified for a GET request. It turns out I was wrong. Removing this line fixes the problem.

The only question I now have is why? I think the content length that is specified is correct, though it's possible it's off by one. Would specifying a content length that is too short prevent the full download from taking place and cause the connection to be left open? I would have expected that Close() and / or Dispose() ought to kill the connection anyway.

Upvotes: 26

Views: 24997

Answers (5)

user330606
user330606

Reputation: 

httpWebRequest.Abort(); // before you leave

Will solve!

Take a look at the resource.

Upvotes: 13

kris164
kris164

Reputation: 11

Plase try use:

         HttpRequestCachePolicy noCachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore);
         httpWebRequest.CachePolicy = noCachePolicy;

Upvotes: 0

Aarón Altamirano
Aarón Altamirano

Reputation: 11

'You need to initialize the webrequest and webresponse objects with a valid URI:

        Dim request As WebRequest = WebRequest.Create("http://google.com")
        Dim response As WebResponse = request.GetResponse()

        Function UrlExists(ByVal URL As String) As Boolean
            Try
                request = WebRequest.Create(URL)
                response = request.GetResponse()
            Catch ex As Exception
                Return False
            Finally
                response.Close()
            End Try
            Return True
        End Function

Upvotes: -1

Jim Mischel
Jim Mischel

Reputation: 134085

Documentation for HttpWebRequest.ContentLength says:

Any value other than -1 in the ContentLength property indicates that the request uploads data and that only methods that upload data are allowed to be set in the Method property.

After the ContentLength property is set to a value, that number of bytes must be written to the request stream that is returned by calling the GetRequestStream method or both the BeginGetRequestStream and the EndGetRequestStream methods.

I suspect it hangs because you're setting the content length but then not uploading any data. One would think that it should throw an exception in that case, but apparently it doesn't.

Upvotes: 1

AdaTheDev
AdaTheDev

Reputation: 147334

Make sure you are creating a NEW HttpWebRequest each time. Quote from the MSDN reference on GetResponse method :

Multiple calls to GetResponse return the same response object; the request is not reissued.

Update 1: Ok. How about if you try closing the webResponseStream too - you currently don't do that, you're only closing the webResponse (just trying to rule things out). I'd also dispose of the httpWebResponse too (not just the WebResponse)

Update 2: The only other thing I can suggest is to have a look at the following articles that I looked at when I did something similar to what you're doing:
http://arnosoftwaredev.blogspot.com/2006/09/net-20-httpwebrequestkeepalive-and.html
http://www.devnewsgroups.net/dotnetframework/t10805-exception-with-httpwebrequest-getresponse.aspx

The only noticeable things I do different are:
- set webrequest.KeepAlive = false
- I don't have the connection management stuff in the config (I don't loop round to make a series of requests)

Upvotes: 8

Related Questions