CODEWITHSUNDEEP
c#asynchronousasync-awaitparallel-foreach
mhaken
mhaken

Reputation: 1125

C# ForEach Loop With ASync Tasks & Dependent Post ASync Tasks

I'm having trouble trying to correctly architect the most efficient way to iterate several async tasks launched from a request object and then performing some other async tasks that depend on both the request object and the result of the first async task. I'm running a C# lambda function in AWS. I've tried a model like this (error handling and such has been omitted for brevity):

public async Task MyAsyncWrapper()
{
  List<Task> Tasks = new List<Task>();
  foreach (var Request in Requests) 
  {
    var Continuation = this.ExecuteAsync(Request).ContinueWith(async x => {
      var KeyValuePair<bool, string> Result = x.Result;
      if (Result.Key == true)
      {
        await this.DoSomethingElseAsync(Request.Id, Request.Name, Result.Value);
        Console.WriteLine("COMPLETED");
      }
    }

    Tasks.Add(Continuation);
  }

  Task.WaitAll(Tasks.ToArray());
}

This approach results in the DoSomethingElseAsync() method not really getting awaited on and in a lot of my Lambda Function calls, I never get the "COMPLETED" output. I've also approached this in this method:

public async Task MyAsyncWrapper()
{
  foreach (var Request in Requests) 
  {
    KeyValuePair<bool, string> Result = await this.ExecuteAsync(Request);

    if (Result.Key == true)
    {
      await this.DoSomethingElseAsync(Request.Id, Request.Name, Result.Value);
      Console.WriteLine("COMPLETED");
    }
  }
}

This works, but I think it's wasteful, since I can only execute one iteration of the loop while waiting on the asnyc's to finish. I also have referenced Interleaved Tasks but the issue is that I basically have two loops, one to populate the tasks, and another to iterate them after they've completed, where I don't have access to the original Request object anymore. So basically this:

List<Task<KeyValuePair<bool, string>>> Tasks = new List<Task<KeyValuePair<bool, string>>>();

foreach (var Request in Requests)
{
  Tasks.Add(ths.ExecuteAsync(Request);
}

foreach (Task<KeyValuePair<bool, string>> ResultTask in Tasks.Interleaved())
{
  KeyValuePair<bool, string> Result = ResultTask.Result;
  //Can't access the original request for this method's parameters
  await this.DoSomethingElseAsync(???, ???, Result.Value);
}

Any ideas on better ways to implement this type of async chaining in a foreach loop? My ideal approach wouldn't be to return the request object back as part of the response from ExecuteAsync(), so I'd like to try and find other options if possible.

Upvotes: 4

Views: 2342

Answers (2)

Richard Szalay
Richard Szalay

Reputation: 84784

I may be misinterpreting, but why not move your "iteration" into it's own function and then use Task.WhenAll to wait for all iterations in parallel.

public async Task MyAsyncWrapper()
{
  var allTasks = Requests.Select(ProcessRequest);

  await Task.WhenAll(allTasks);
}

private async Task ProcessRequest(Request request)
{
    KeyValuePair<bool, string> Result = await this.ExecuteAsync(request);

    if (Result.Key == true)
    {
      await this.DoSomethingElseAsync(request.Id, request.Name, Result.Value);
      Console.WriteLine("COMPLETED");
    }
}

Upvotes: 3

Cory Nelson
Cory Nelson

Reputation: 30031

Consider using TPL dataflow:

var a = new TransformBlock<Input, OutputA>(async Input i=>
{
    // do something async.
    return new OutputA();
});

var b = new TransformBlock<OutputA, OutputB>(async OutputA i =>
{
    // do more async.
    return new OutputB();
});

var c = new ActionBlock<OutputB>(async OutputB i =>
{
    // do some final async.
});

a.LinkTo(b, new DataflowLinkOptions { PropogateCompletion = true });
b.LinkTo(c, new DataflowLinkOptions { PropogateCompletion = true });

// push all of the items into the dataflow.
a.Post(new Input());
a.Complete();

// wait for it all to complete.
await c.Completion;

Upvotes: 3

Related Questions