CODEWITHSUNDEEP
c#parallel-processing
Scott
Scott

Reputation: 13921

Parallel programming for Windows Service

I have a Windows Service that has code similar to the following:

List<Buyer>() buyers = GetBuyers();
var results = new List<Result();

Parallel.Foreach(buyers, buyer =>
{
    // do some prep work, log some data, etc.

    // call out to an external service that can take up to 15 seconds each to return
    results.Add(Bid(buyer));    
}

// Parallel foreach must have completed by the time this code executes
foreach (var result in results)
{
    // do some work
}

This is all fine and good and it works, but I think we're suffering from a scalability issue. We average 20-30 inbound connections per minute and each of those connections fire this code. The "buyers" collection for each of those inbound connections can have from 1-15 buyers in it. Occasionally our inbound connection count sees a spike to 100+ connections per minute and our server grinds to a halt.

CPU usage is only around 50% on each server (two load balanced 8 core servers) but the thread count continues to rise (spiking up to 350 threads on the process) and our response time for each inbound connection goes from 3-4 seconds to 1.5-2 minutes.

I suspect the above code is responsible for our scalability problems. Given this usage scenario (parallelism for I/O operations) on a Windows Service (no UI), is Parallel.ForEach the best approach? I don't have a lot of experience with async programming and am looking forward to using this opportunity to learn more about it, figured I'd start here to get some community advice to supplement what I've been able to find on Google.

Upvotes: 3

Views: 911

Answers (1)

usr
usr

Reputation: 171178

Parallel.Foreach has a terrible design flaw. It is prone to consume all available thread-pool resources over time. The number of threads that it will spawn is literally unlimited. You can get up to 2 new ones per second driven by heuristics that nobody understands. The CoreCLR has a hill climbing algorithm built into it that just doesn't work.

call out to an external service

Probably, you should find out what's the right degree of parallelism calling that service. You need to find out by testing different amounts.

Then, you need to restrict Parallel.Foreach to only spawn as many threads as you want at a maximum. You can do that using a fixed concurrency TaskScheduler.

Or, you change this to use async IO and use SemaphoreSlim.WaitAsync. That way no threads are blocked. The pool exhaustion is solved by that and the overloading of the external service as well.

Upvotes: 3

Related Questions