Concurrently running multiple tasks C#

Question

I have REST web API service in IIS which takes a collection of request objects. The user can enter more than 100 request objects.

I want to run this 100 request concurrently and then aggregate the result and send it back. This involves both I/O operation (calling to backend services for each request) and CPU bound operations (to compute few response elements)

Code snippet -

using System.Threading.Tasks;
....

var taskArray = new Task<FlightInformation>[multiFlightStatusRequest.FlightRequests.Count];

for (int i = 0; i < multiFlightStatusRequest.FlightRequests.Count; i++)
{
    var z = i;
    taskArray[z] = Tasks.Task.Run(() =>
        PerformLogic(multiFlightStatusRequest.FlightRequests[z],lite, fetchRouteByAnyLeg)
        );
}
Task.WaitAll(taskArray);
for (int i = 0; i < taskArray.Length; i++)
{
    flightInformations.Add(taskArray[i].Result);
}

public Object PerformLogic(Request,...)
{
    //multiple IO operations each depends on the outcome of the previous result
    //Computations after getting the result from all I/O operations
}

If i individually run the PerformLogic operation (for 1 object) it is taking 300 ms, now my requirement is when I run this PerformLogic() for 100 objects in a single request it should take around 2 secs.

PerformLogic() has the following steps - 1. Call a 3rd Party web service to get some details 2. Based on the details call another 3rd Party webservice 3. Collect the result from the webservice, apply few transformation

But with Task.run() it takes around 7 secs, I would like to know the best approach to handle concurrency and achieve the desired NFR of 2 secs. I can see that at any point of time 7-8 threads are working concurrently

not sure if I can spawn 100 threads or tasks may be we can see some better performance. Please suggest an approach to handle this efficiently.

Answer 1

Judging by this

public Object PerformLogic(Request,...)
{
  //multiple IO operations each depends on the outcome of the previous result
  //Computations after getting the result from all I/O operations
}

I'd wager that PerformLogic spends most its time waiting on the IO operations. If so, there's hope with async. You'll have to rewrite PerformLogicand maybe even the IO operations - async needs to be present in all levels, from the top to the bottom. But if you can do it, the result should be a lot faster.

Other than that - get faster hardware. If 8 cores take 7 seconds, then get 32 cores. It's pricey, but could still be cheaper than rewriting the code.

Answer 2

First, don't reinvent the wheel. PLINQ is perfectly capable of doing stuff in parallel, there is no need for manual task handling or result merging.

If you want 100 tasks each taking 300ms done in 2 seconds, you need at least 15 parallel workers, ignoring the cost of parallelization itself.

var results = multiFlightStatusRequest.FlightRequests
                                      .AsParallel()
                                      .WithDegreeOfParallelism(15) 
                                      .Select(flightRequest => PerformLogic(flightRequest, lite, fetchRouteByAnyLeg)
                                      .ToList();

Now you have told PLinq to use 15 concurrent workers to work on your queue of tasks. Are you sure your machine is up to the task? You could put any number you want in there, that doesn't mean that your computer magically gets the power to do that.

Another option is to look at your PerformLogic method and optimize that. You call it 100 times, maybe it's worth optimizing.