CODEWITHSUNDEEP
c#.net-4.0asynchronoushttpwebrequest
user17753
user17753

Reputation: 3161

blocking main thread until all async calls are done

This question had an excellent answer on how to perform an asynchronous HttpWebRequest.

I have a list of 50k+ URLs that I want to asynchronously fetch with this method. The issue I ran into is the main thread would complete and exit before all the async requests could finish.

I found ManualResetEvent could be used to WaitOne(), and to call Set() on the call back method. This would work great if I was only doing one request. Not sure how to handle this for many requests.

Upvotes: 0

Views: 950

Answers (1)

Darin Dimitrov
Darin Dimitrov

Reputation: 1038720

You could use TPL:

using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Threading.Tasks;

class Program
{
    static void Main(string[] args)
    {
        var urls = new[] 
        { 
            "http://www.google.com", 
            "http://www.yahoo.com" 
        };

        var tasks = urls.Select(url =>
        {
            var request = WebRequest.Create(url);
            return Task
                .Factory
                .FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, url)
                .ContinueWith(t =>
                {
                    if (t.IsCompleted)
                    {
                        using (var stream = t.Result.GetResponseStream())
                        using (var reader = new StreamReader(stream))
                        {
                            Console.WriteLine("-- Successfully downloaded {0} --", t.AsyncState);
                            Console.WriteLine(reader.ReadToEnd());
                        }
                    }
                    else if (t.IsFaulted)
                    {
                        Console.WriteLine("There was an error downloading {0} - {1}", t.AsyncState, t.Exception);
                    }
                });
        }).ToArray();

        Task.WaitAll(tasks);
    }
}

Upvotes: 1

Related Questions