What is the behaviour of await inside of a Parallel.ForEach() loop?

Question

I have a computationally intensive program which I am attempting to parallelize, however one of the limiting steps is an I/O operation which is controlled by a phenomenally inefficient API I have no control over but have no choice but to use. It is imperative that my parallelization does not increase the number of I/O operations, or any benefit will likely very quickly disappear.

The layout is something like this: I have two classes, Foo and Bar, and in order to calculate Foo, which involves no small quantity of calculations, I must pass it an instance, or a few instances, of Bar which I import from some other file in an extremely expensive I/O operation. I require a large number of both Foo and Bar instances and many of these Bar instances will be used to calculate more than one Foo instance. As a result, I do not want to discard my Bar instances after I calculate each Foo and I do not want to import them more than once each. Potentially of note, to make matters more complicated the API is 32-bit, whereas my program must be 64-bit to avoid MemoryException, so that is handled by a locally hosted server which I communicate with using WCF.

Here is my proposed solution, but I am extremely new to parallelization and in particular I am unsure of how the await will be handled inside of the ForEach loop w.r.t freeing up processors:

ConcurrentDictionary<string, Task<Bar>> barList = new ConcurrentDictionary<string, Task<Bar>>();

Parallel.ForEach(fooList, foo =>
{
    if (!barList.ContainsKey(this.RequiredBarName))
    {
        Task<Bar> importBar = Task.Run(() => Import.BarByName(this.RequiredBarName));
        barList.Add(this.RequiredBarName,importBar);
    }
    this.RequiredBarTask = barList.TryGetValue(this.RequiredBarName);
    foo.CalculateStuff();
}

// where foo.CalculateStuff() looks something like this
async public void CalculateStuff()
{
    // do some stuff...
    Bar requiredBar = await this.RequiredBarTask;
    // do some more stuff with requiredBar
}

What will happen when the code runs into that await? Will the ThreadPool pick up a different Task, or will the processor just idle? If I then arrange some sort of WaitAll() outside of the Parallel.ForEach() will I be able to parallelize through all of this efficiently? Does anyone have any better ideas of how I might implement this?

Edit to provide MCVE:

I cannot satisfy the Verifiable component of this as I cannot give you the API and I certainly can't give you any of the data that the API might access, however I will attempt to provide you with something up to the call out to the server.

The program can effectively go infinitely deep in the way it processes things, it is much easier to think of as a parser of specific instructions which the client is allowed to build using the GUI an a set of "bricks". In this way Dataflow looks like it could offer a decent solution.

In this example I don't take care of circular references or one Channel calculating another Channel which has already been called for by the Parallel.ForEach() method; in my code this is handled by some logic and Concurrent lists to check when various things have been called.

public abstract class Class
{
    public string Name {get;set;}
    public float[] Data {get;set;}

    async public Task CalculateData(IsampleService proxy){}
}

public class Channel : Class
{
    public Class[] ChildClasses {get;set;}

    async public override Task CalculateData(IsampleService proxy)
    {
        foreach(Class childClass in ChildClasses)
        {
            // not the real processing but this step could be anything. There is a class to handle what happens here, but it is unnecessary for this post.
            if(childClass.Data==null) await childClass.CalculateData(proxy);
            this.Data = childClass.Data;
        }
    }
}

public class Input : Class
{
    async public override Task CalculateData(IsampleService proxy)
    {
            this.Data = await proxy.ReturnData(this.Name);
    }
}

async public static Task ProcessDataForExport(Channel[] channelArray)
{
ChannelFactory<IsampleService> factory = new ChannelFactory<IsampleService>(new NetNamedPipeBinding(), new EndpointAddress(baseAddress));

IsampleService proxy = factory.CreateChannel();

Parallel.ForEach(channelArray, channel =>
    {
        channel.CalculateData();
    });
// Task.WhenAll() might be a better alternative to the Parallel.ForEach() here.
}

Answer 1

What will happen when the code runs into that await?

The same thing that happens for any await statement: after having evaluated whatever expression or statement retrieves the Task to be awaited, the method will return. For all intents and purposes, that is the end of the method.

Will the ThreadPool pick up a different Task, or will the processor just idle?

That depends on what else is going on. For example, what are you awaiting on? If it's a computational task queued to the thread pool, and it wasn't already assigned a thread pool thread, then sure…the thread pool might pick that up and start working on it.

If you're waiting on an I/O operation, then that won't necessarily keep the processor busy, but there may still be other tasks in the thread pool queue (such as other ones from the Parallel.ForEach() call). So that would give the processor something to work on.

Certainly, using await doesn't generally result in the processer being idle. In fact, the main reason for using it is to avoid just that (*). As the await statement causes the current method to return, you let the current thread proceed, which means that if otherwise there weren't enough threads to keep the processor busy, now it has something to do. :)

(*) (well, sort of…really, the main reason is to avoid blocking the current thread, but that has the side-effect of there being more work available for the processer to handle :) )

If I then arrange some sort of WaitAll() outside of the Parallel.ForEach() will I be able to parallelize through all of this efficiently? Does anyone have any better ideas of how I might implement this?

I don't see enough useful detail in your question to answer that. Frankly, while I can't put my finger on it, the use of await from a Parallel.ForEach() delegate seems fishy to me somehow. As soon as you call await, the delegate's method will return.

Hence, as far as Parallel.ForEach() knows, you're done with that item in the enumeration, but of course you're not. It will have to be finished elsewhere. At the very least, that seems like it would hinder the Parallel class's ability to know enough about the work it's doing to schedule it most effectively.

But maybe that's okay. Or maybe it's not great, but is the best you're going to achieve given the framework you're tied to. Hard to say.

I do encourage you to provide the MCVE that commenter Scott Chamberlain's asked for. If he's right and your problem is addressable through the dataflow API, you would do well to give him the chance to provide you an answer that shows that.