CODEWITHSUNDEEP
c#.netasp.net-web-apitask-parallel-library.net-4.5
Erick T
Erick T

Reputation: 7449

Why does this code fail when executed via TPL/Tasks?

I am using System.Net.Http to use network resources. When running on a single thread it works perfectly. When I run the code via TPL, it hangs and never completes until the timeout is hit.

What happens is that all the threads end up waiting on the sendTask.Result line. I am not sure what they are waiting on, but I assume it is something in HttpClient.

The networking code is:

using (var request = new HttpRequestMessage(HttpMethod.Get, "http://google.com/"))
{
    using (var client = new HttpClient())
    {
        var sendTask = client.SendAsync
              (request, HttpCompletionOption.ResponseHeadersRead);
        using (var response = sendTask.Result)
        {
            var streamTask = response.Content.ReadAsStreamAsync();
            using (var stream = streamTask.Result)
            {
                // problem occurs in line above
            }
        }
    }
}

The TPL code that I am using is as follows. The Do method contains exactly the code above.

var taskEnumerables = Enumerable.Range(0, 100);
var tasks = taskEnumerables.Select
            (x => Task.Factory.StartNew(() => _Do(ref count))).ToArray();
Task.WaitAll(tasks);

I have tried a couple of different schedulers, and the only way that I can get it to work is to write a scheduler that limits the number of running tasks to 2 or 3. However, even this fails sometimes.

I would assume that my problem is in HttpClient, but for the life of me I can't see any shared state in my code. Does anyone have any ideas?

Thanks, Erick

Upvotes: 2

Views: 620

Answers (1)

Erick T
Erick T

Reputation: 7449

I finally found the issue. The problem was that HttpClient issues its own additional tasks, so a single task that I start might actually end spawning 5 or more tasks.

The scheduler was configured with a limit on the number of tasks. I started the task, which caused the number of running tasks to hit the max limit. The HttpClient then attempted to start its own tasks, but because the limit was reached, it blocked until the number of tasks went down, which of course never happened, as they were waiting for my tasks to finish. Hello deadlock.

The morals of the story:

  1. Tasks might be a global resource
  2. There are often non-obvious interdependencies between tasks
  3. Schedulers are not easy to work with
  4. Don't assume that you control either schedulers or number of tasks

I ended up using another method to throttle the number of connections.

Erick

Upvotes: 2

Related Questions