HttpWebRequest.GetResponse() hangs randomly

Question

I have the following code :

public static QHttpResponse Execute(QHttpRequest request)
{
    //Setup the request
    HttpWebRequest webrequest = (HttpWebRequest) WebRequest.Create(request.GetFinalUrl());
    webrequest.AllowAutoRedirect = request.IsAllowRedirects;
    webrequest.Method = request.Method;
    webrequest.Accept = "application/json, text/javascript;q=0.9, */*;q=0.5";
    webrequest.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
    //request.Host is set automatically
    webrequest.UserAgent = request.UserAgent;
    if (!String.IsNullOrEmpty(request.Referrer))
        webrequest.Referer = request.Referrer;
    webrequest.Timeout = 50000;
    webrequest.KeepAlive = false;
    webrequest.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Revalidate);
    webrequest.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
    if (request.IsAjax)
    {
        webrequest.Headers.Add("X-Request", "JSON");
        webrequest.Headers.Add("X-Requested-With", "XMLHttpRequest");
        webrequest.Headers.Add("X-Prototype-Version", "1.7");
    }

    // Cookies
    webrequest.CookieContainer = request.Cookies;

    // Write the data to the body for POST and other methods
    if (request.IsMethodPost())
    {
        byte[] dataBytes = Encoding.UTF8.GetBytes(request.GetDataParamString());
        webrequest.ContentLength = dataBytes.Length;

        using (Stream requestStream = webrequest.GetRequestStream())
            requestStream.Write(dataBytes, 0, dataBytes.Length);
    }

    // Get the response
    HttpWebResponse webresponse;
    try
    {
        webresponse = webrequest.GetResponse() as HttpWebResponse;
    }
    catch (WebException wex)
    {
        if(request.IsBypassError)
            webresponse = wex.Response as HttpWebResponse;
        else
            throw;
    }

    // Read to QHttpResponse object
    QHttpResponse response = new QHttpResponse();
    response.StatusCode = webresponse.StatusCode;
    response.NewCookies = webresponse.Cookies;
    using (Stream responseStream = webresponse.GetResponseStream())
        using (StreamReader reader = new StreamReader(responseStream))
            response.Reply = reader.ReadToEnd();

    webresponse.Close();

    return response;
}

I have this code run multiple times from various locations and randomly (Every couple of hours), it hangs at this line :

webresponse = webrequest.GetResponse() as HttpWebResponse;

I tried setting webrequest.KeepAlive = false;, but I continued to receive the error.

I'd like any available help on solving this, thanks in advance.

EDIT : I'd like to add that this code is executed from two threads. Occasionally they may connect to the same host, but only from these 2 threads. Also, as I see, the response is closed appropriately.

EDIT 2 : Visual studio's debugger says the execution is really stuck at System.dll!System.Net.Sockets.Socket.Receive.

EDIT 3 : In an attempt to see exactly what was causing the bug, I modified the "Get the response" code from above to

// Get the response
HttpWebResponse webresponse = null;
try
{
    webresponse = webrequest.GetResponse() as HttpWebResponse;
}
catch (WebException wex)
{
    Console.WriteLine("Time : " + DateTime.Now);
    Console.WriteLine("Thread name : " + Thread.CurrentThread.Name);
    Console.WriteLine("Exception : " + wex);
    Console.WriteLine("Exc msg : " + wex.Message);
    Console.WriteLine("Url : " + request.GetFinalUrl());

    if (request.IsBypassError)
        webresponse = wex.Response as HttpWebResponse;
    else
    {
        if (webresponse != null)
        {
            webresponse.Close();
            webresponse.Dispose();
        }

        throw;
    }
}

I received this output :

Time : 5/11/2015 3:13:35 AM 
Thread name : BOT A
Exception : System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
at System.Net.HttpWebRequest.GetResponse()
at Gameloop.Util.Web.QWebClient.Execute(QHttpRequest request) in e:\Visual Studio - Workspace\Gameloop.Util\Gameloop.Util\Web\QWebClient.cs:line 52 
Exc msg : The remote server returned an error: (500) Internal Server Error. 
Url : https://website1.com/url/path/to/something (I changed this)

This was the only displayed error and was encountered by thread "BOT A". However, this was not the url the threads appear to have actually frozen at. "BOT A" was actually frozen at 12:00pm at website2.com and "BOT B" was actually frozen at 7:00am at website3.com. I doubt the hanging has much to do with the exception since the requests would have been made a large number of times after that before the actual hang.

Answer 1

Just add below in the try block:

httpWReq.Timeout = 3000;

Answer 2

My first inclination is that you may need to dispose of your HttpWebResponse. Normally you might wrap that in a using block, but since you have two places where webresponse might be assigned, you might just want to dispose it explicitly, like this.

webresponse.Close();
webresponse.Dispose();

I would start there.