How can I get HttpWebRequest to behave as synchronously as possible?

Question

---

Summary: Is this even possible?

unit test function on ui thread:
- creates a background thread
- starts the thread
- waits for it to complete (function does not exit until it completes!)

background thread:
- opens an http stream 
- reads a url from the web
- terminates

My suspicion: The framework asynchronously puts the result onto some internal message queue and thus the response callback will never be called until the ui thread's stack unwinds and goes to some ui thread function to pump the stack.

---

The full story:

I am porting an app that requires creating a stream from various sources, one of them being from a simple http url. I am doing this on a background thread, and ideally I would like it to behave 100% synchronously, just block when needed (this is ok since it's on a background thread).

But it seems the framework is a bit mickey mouse in that it assumes you will be doing the request on the ui thread and so it will shield the coder from having to create a background thread to do the asynch operation. But I may be missing something.

I have stumbled across the following article: http://pieterderycke.wordpress.com/2011/05/23/adding-synchronous-methods-to-webrequest-on-windows-phone-7/, which suggests a solution to make the http web request synchronous. But as it is implemented, I get a ProtocolViolationException. I have since made a modification to the code to use BeginGetResponse() instead of BeginGetRequestStream(), and this seems to no longer cause the exception.

But it seems that the background thread now blocks indefinitely. On my ui thread I loop, doing a Thread.Sleep(10) since I am in a unit test function, waiting for my callback to get called. Is it possible that the callback will not be called until unit test function returns and the ui thread has a chance to pump messages? If so, any way I can force it to pump so that I can continue where I left off in the unit test routine?

At the bottom of the article mentioned above, a comment is made "If you test your code, you will find that it deadlocks if you execute it on the UI thread." but I am executing it on a background thread, so should be ok right?

The msdn docs only show you how to do asynch calls. And they also mention that "The BeginGetResponse method requires some synchronous setup tasks to complete" ... "typically several seconds" ... but "can take 60 seconds or more". This sounds pretty bad to be executed on a ui thread. http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse.aspx

Please help!

Here is my code:

using System.Net;
using System.Threading;
using System;
using System.IO;

namespace Blah
{
  // http://pieterderycke.wordpress.com/2011/05/23/adding-synchronous-methods-to-webrequest-on-windows-phone-7/
  // Creates synchronous web requests.
  // Must not be called on UI threads.

  public static class WebRequestExtensions
  {
    public static Stream GetRequestStream(this WebRequest request)
    {
      AutoResetEvent autoResetEvent = new AutoResetEvent(false);

      IAsyncResult asyncResult = null;
      {
        // http://stackoverflow.com/questions/253549/how-do-i-use-httpwebrequest-with-get-method

        if (request.Method == "GET")
        {
          asyncResult = request.BeginGetResponse(
           r => autoResetEvent.Set(), null);
        }
        else
        {
          asyncResult = request.BeginGetRequestStream(
           r => autoResetEvent.Set(), null);
        }
      }

      // Wait until the call is finished
      autoResetEvent.WaitOne();

      return request.EndGetRequestStream(asyncResult);
    }
  }
}

I've also recently stumbled across http://www.eggheadcafe.com/tutorials/aspnet/91f69224-3da5-4959-9901-c5c717c9b184/making-silverlight-emulate-synchronous-requests.aspx, but this exhibits the same problem. Seems that I don't get my callback until the ui thread returns up the stack... I can smell some sort of a framework message queue there somewhere, am I correct?

Thanks

Answer 1

While I'm not advocating moving stuff to the UI thread, which is what you're probably going to end up with, there are times when porting synchronous code to the Phone that it would be nice to have a synchronous version of GetResponse. I wrote these extensions methods for a library I maintain that's compatible with multiple platforms.

Answer 2

As HTTP response processing utilises the UI thread, blocking it will prevent the request from completing.

Assuming you are using the Silverlight Unit Testing Framework bits, you can mark your test as [Asynchronous]:

using System;
using System.Net;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Microsoft.Silverlight.Testing;

[TestClass]
public class WebRequestsTests : WorkItemTest
{
    [TestMethod, Asynchronous]
    public void TestWebRequest()
    {
        var webRequest = WebRequest.CreateHttp("http://www.stackoverflow.com");

        webRequest.BeginGetResponse(result =>
        {
            EnqueueCallback(() =>
            {
                WebResponse response = webRequest.EndGetResponse(result);

                // process response 

                TestComplete(); // async test complete 
            });
        }, null);
    }
} 

Alternatively, if you're looking to use Rx (which I do, personally), I recently did a blog post on how to make Rx-based asynchronous tests cleaner when using the SL testing framework.

Answer 3

Why ask for a bad solution to the wrong problem? If you need continuations on asynchronous requests, you should simply use the Task Parallel Library.

You can find a version of TPL for Windows Phone on NuGet: Task Parallel Library for Silverlight.

This allows continuations, which essentially would give the same effect as blocking your thread.

(Robert McLaws, creator of the library) NuGet is a system that makes it easy to manage dependencies. Just go to http://nuget.org/, click the link to install it, then open the Package Manager Console and type "Install-Package System.Threading.Tasks", and it will automatically install the right version for your project. I am not currently making the source available, although I might sometime in the future. I need to check with the Mono guys, as the code comes mostly from Mono.

Answer 4

you could also use Rx (that ships with the library) to do this.

http://dotnet.dzone.com/articles/5-minute-sample-fromasyncpattern