CODEWITHSUNDEEP
c#memory-managementmemory-leaksgarbage-collection
ren
ren

Reputation: 3993

Unexpected OutOfMemoryException

I used to have:

using (MyWebClient client = new MyWebClient(TimeoutInSeconds))
{
   var res = client.DownloadData(par.Base_url);
   //code that checks res
}

Now I have:

using (MyWebClient client = new MyWebClient(TimeoutInSeconds))
{
   client.DownloadDataAsync(new Uri(par.Base_url));
   client.DownloadDataCompleted += (sender, e) =>
   {
       //code that checks e.Result
   }
}

Where MyWebClient is derived from WebClient. Now I have lots of threads doing this and in the first case memory consumption wasn't an issue while in the second one I see steady rise in memory until I get OutOfMemoryException. I profiled and it seems that WebClient is the culprit, not being disposed and downloaded data is kept. But why? What's the difference between two cases? Perhaps e.Result needs to be somehow disposed of?

Upvotes: 0

Views: 486

Answers (2)

Adam Robinson
Adam Robinson

Reputation: 185693

You are disposing of your WebClient immediately in the second option. You have a couple of choices:

  1. If you're using .NET 4.5 (or .NET 4.0 with Visual Studio 2012 and the AsyncTargetingPack installed), you can do var res = await client.DownloadDataAsync(par.Base_url); and have code that looks similar to your first line but is actually asynchronous.
  2. Use a normal continuation and get rid of your using block

The first option would look like this:

using (MyWebClient client = new MyWebClient(TimeoutInSeconds))
{
   var res = await client.DownloadDataAsync(par.Base_url);
   //code that checks res
}

The second option would look like this:

var client = new MyWebClient(TimeoutInSeconds);

client.DownloadDataAsync(new Uri(par.Base_url))
    .ContinueWith(t =>
    {
        client.Dispose();

        var res = t.Result;

        //code that checks res
    }
}

HOWEVER

You must change your threading approach depending on which solution you use. The first version of your code runs synchronously, so if you have a thread dedicated to a URL (or connection or however it is you're splitting them up), the downloads will run synchronously on that thread and block it. If you choose either of these options, however, you'll end up using IO completion threads to complete your work, splitting it out from the main thread. In the long run, this is probably better, but it means you have to be mindful about how many of these requests you submit in parallel.

Upvotes: 0

David Schwartz
David Schwartz

Reputation: 182837

Your first case limits the number of concurrent downloads to the number of threads. Your second case has no limit on the number of concurrent downloads.

Upvotes: 2

Related Questions