CODEWITHSUNDEEP
c#multithreadingtask-parallel-libraryparallel.foreach
Giorgi Nakeuri
Giorgi Nakeuri

Reputation: 35780

Ordered parallel execution

I have an ordered list like [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. I am passing it to a Parallel.ForEach statement. Can I somehow achieve the following ordering of execution of buckets like: process first 3 items [1, 2, 3] where ordering in bucket itself is not mandatory and can be [2, 1, 3] for instance. Then process next 3 items [4, 5, 6], etc?

Upvotes: 4

Views: 788

Answers (2)

VMAtm
VMAtm

Reputation: 28355

Even if the accepted answer fulfills requirements perfectly, there is some overhead in it. First of all, as we are talking about the TPL, the volume of data arrays is probably big, so simply creating that much arrays is very memory consuming.
Also, solution suggested by @viveknuna does not guarantee the order for the chunks. If it is ok, you probably should use the answer from @DmitryBychenko with a small update:

const int chunkSize = 3;
var array = Enumerable.Range(1, 9).ToArray();
// get the chunks for indexes for array sized in group of 3
var partitioner = Partitioner.Create(0, array.Length, chunkSize);

// use all the system resources
var parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount };

// use the partitioner for a chunks, so outer parallel foreach
// will start a task for all the chunks, [1, 2, 3], [4, 5, 6], [7, 8, 9]
Parallel.ForEach(partitioner, parallelOptions, part =>
    {
        // inner foreach will handle part of the chunk in parallel
        Parallel.ForEach(array.Skip(part.Item1).Take(chunkSize), parallelOptions, value =>
            {
                // handle the array value in parallel
            });
    });

In the given code, if you set the for the ParallelOptions.MaxDegreeOfParallelism to the 1, you'll get the desired ordered parallel execution, chunk by chunk.

Upvotes: 2

Vivek Nuna
Vivek Nuna

Reputation: 1

I'm not sure that you can do this directly. but I would suggest you to divide the input list in to smaller lists and then you can process each sublist with Parallel.Foreach.

List<int> fruits = new List<int>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
List<List<int>> ls = new List<List<int>>();
for (int i = 0; i < fruits.Count; i += 3)
{
    ls.Add(fruits.GetRange(i, Math.Min(3, fruits.Count - i)));
}
foreach (List<int> group in ls)
{
    Parallel.ForEach(group, fruit =>
    {
    });
}

3 is the length of small list.

Upvotes: 5

Related Questions