CODEWITHSUNDEEP
c#multithreadingparallel-extensions
jasper
jasper

Reputation: 3474

.net Parallel and Serial looping

We would like to build a pattern around certain loops in our solution, that will allow them to run in serial, or parallel, depending on factors. Below is the general form of it.

Since the concurrent collections dont share a common interface with the regular collections, we need some sort of adapter to write general code.

Specifically around the usage of the addFunc delegate in the loop body, is there anything there that would end up causing problems in the long term that we might miss? It runs fine as of now, but....?

Action<SomeType> addFunc;

if(runInParallel)
{
   addFunc = concurrentBag.Add;
   loopDelegate = Parallel.ForEach;
}
else
{
   addFunc = iList.Add;
   loopDelegate = Serial.ForEach; // wrapper delegate for foreach
}

loopDelegate(source, item =>
{
   SomeType result = longRunningTask(item);
   ...
   addFunc(result); // will this 
});

Upvotes: 3

Views: 687

Answers (1)

M Afifi
M Afifi

Reputation: 4795

Curious why not use TPL in .NET 4.0? http://msdn.microsoft.com/en-us/library/dd537609.aspx

There is an excellent white paper of the considerations they have taken when developing TPL, if you can't use .NET 4, you should look at the paper and consider some of the gotcha's in there.

Updated based on comment pointing out the obvious.

I'd use some syntactic sugar like,

ForEach<Tsource>(Predicate<IEnumerable<TSource>> isParallel, IEnumerable<TSource> source, Action<TSource> body)
{
    if(isParallel(source))
    {
        Parallel.ForEach<TSource>(source, body);
    }
    else
    {
        foreach (TSource element in source)
        {
            body(element);
        }
    }
}

There are two main advantages over your implementation.

  1. You enumerate twice, once to add the items to loop over, and a 2nd time during execution.
  2. It is not immediately obvious, but you're stopping Parallel.ForEach, and foreach from using the most efficient getter. Parallel.ForeEach will not always use GetEnumerator in IEnumerable. An enumerator by design is not concurrent, if your item implements IList, Parallel.ForEach will use the indexer to allow each thread to access the element in the source without waiting on the enumerator to iterate through the list. ConcurrentBag does not implement IList.

This is the paper I was referring to btw, http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=19222.

It's all a very good read if you're imeplemeting this, but pay particular attention to, Pages 1[5-7], 26, 3[0-2].

The syntactic sugar means you can call it as you would with TPL,

MyParllelLibrary.ForEach( (list) => true, list), item =>
{
    // What my code does
});

Upvotes: 2

Related Questions