Multiple parallel Tasks in C# do not improve calculation time

Question

I have a complicated math problem to solve and I decided to do some independent calculations in parallel to improve calculation time. In many CAE programs, like ANSYS or SolidWorks, it is possible to set multiple cores for that purpose.

I created a simple Windows Form example to illustrate my problem. Here the function CalculateStuff() raises A from Sample class in power 1.2 max times. For 2 tasks it's max / 2 times and for 4 tasks it's max / 4 times.

I calculated the resulting time of operation both for only one CalculateStuff() function or four duplicates (CalculateStuff1(), ...2(), ...3(), ...4() - one for each task) with the same code. I'm not sure, if it matters to use the same function for each task (anyway, Math.Pow is the same). I also tried to enable or disable the ProgressBar.

The table represents time of operation (sec) for all 12 cases. I expected it to be like 2 and 4 times faster for 2 and 4 tasks, but in some cases 4 tasks are even worse than 1. My computer has 2 processors, 10 cores each. According to Debug window, CPU usage increases with more tasks. What's wrong with my code here or do I misunderstand something? Why multiple tasks do not improve time of operation?

        private readonly ulong max = 400000000ul;

        // Sample class
        private class Sample
        {
            public double A { get; set; } = 1.0;
        }

        // Clear WinForm elements
        private void Clear()
        {
            PBar1.Value = PBar2.Value = PBar3.Value = PBar4.Value = 0;
            TextBox.Text = "";
        }

        // Button that launches 1 task
        private async void BThr1_Click(object sender, EventArgs e)
        {
            Clear();
            DateTime start = DateTime.Now;

            Sample sample = new Sample();

            await Task.Delay(100);
            Task t = Task.Run(() => CalculateStuff(sample, PBar1, max));
            await t;

            TextBox.Text = (DateTime.Now - start).ToString(@"hh\:mm\:ss");

            t.Dispose();
        }

        // Button that launches 2 tasks
        private async void BThr2_Click(object sender, EventArgs e)
        {
            Clear();
            DateTime start = DateTime.Now;

            Sample sample1 = new Sample();
            Sample sample2 = new Sample();

            await Task.Delay(100);
            Task t1 = Task.Run(() => CalculateStuff(sample1, PBar1, max / 2));
            Task t2 = Task.Run(() => CalculateStuff(sample2, PBar2, max / 2));
            await t1; await t2;

            TextBox.Text = (DateTime.Now - start).ToString(@"hh\:mm\:ss");

            t1.Dispose(); t2.Dispose();
        }

        // Button that launches 4 tasks
        private async void BThr4_Click(object sender, EventArgs e)
        {
            Clear();
            DateTime start = DateTime.Now;

            Sample sample1 = new Sample();
            Sample sample2 = new Sample();
            Sample sample3 = new Sample();
            Sample sample4 = new Sample();

            await Task.Delay(100);
            Task t1 = Task.Run(() => CalculateStuff(sample1, PBar1, max / 4));
            Task t2 = Task.Run(() => CalculateStuff(sample2, PBar2, max / 4));
            Task t3 = Task.Run(() => CalculateStuff(sample3, PBar3, max / 4));
            Task t4 = Task.Run(() => CalculateStuff(sample4, PBar4, max / 4));
            await t1; await t2; await t3; await t4;

            TextBox.Text = (DateTime.Now - start).ToString(@"hh\:mm\:ss");

            t1.Dispose(); t2.Dispose(); t3.Dispose(); t4.Dispose();
        }

        // Calculate some math stuff
        private static void CalculateStuff(Sample s, ProgressBar pb, ulong max)
        {
            ulong c = max / (ulong)pb.Maximum;

            for (ulong i = 1; i <= max; i++)
            {
                s.A = Math.Pow(s.A, 1.2);

                if (i % c == 0)
                    pb.Invoke(new Action(() => pb.Value = (int)(i / c)));
            }
        }

Answer 1

There are a lot of possible reasons that you might not see the performance gains you're expecting, including things like what else your machine's cores are getting used for at the moment. Running this trimmed-down version of your code, I am able to see a marked improvement when running parallel:

private IEnumerable<Sample> CalculateMany(int n)
{
    return Enumerable.Range(0, n)
        .AsParallel() // comment this to remove parallelism
        .Select(i => { var s = new Sample(); CalculateStuff(s, max / (ulong)n); return s; })
        .ToList();
}

// Calculate some math stuff
private static void CalculateStuff(Sample s, ulong max)
{
    for (ulong i = 1; i <= max; i++)
    {
        s.A = Math.Pow(s.A, 1.2);
    }
}

Here's running CalculateMany with n values as 1, 2, and 4:

Here's what I get if not using parallelism:

I see similar results using Task.Run():

private IEnumerable<Sample> CalculateMany(int n)
{
    var tasks = 
    Enumerable.Range(0, n)
        .Select(i => Task.Run(() => { var s = new Sample(); CalculateStuff(s, max / (ulong)n); return s; }))
        .ToArray()        ;
    Task.WaitAll(tasks);
    return tasks
        .Select(t => t.Result)
        .ToList();
}

Answer 2

Tasks are not threads. "Asynchronous" does not mean "simultaneous".

What's wrong with my code here or do I misunderstand something?

You're misunderstanding what tasks are.

You should think of tasks as something that you can do in any order you desire. Take the example of a cooking recipe:

• Cut the potatoes
• Cut the vegetables
• Cut the meat

If these were not tasks and it was synchronous code, you would always do these steps in the exact order they were listed.

If they were tasks, that doesn't mean these jobs will be done simultaneously. You are only one person (= one thread), and you can only do one thing at a time.
You can do the tasks in any order you like, you can possibly even halt one task to begin on another, but you still can't do more than one thing at the same time. Regardless of the order in which you complete the tasks, the total time taken to complete all three tasks remains the same, and this is not (inherently) any faster.

If they were threads, that's like hiring 3 chefs, which means these jobs can be done simultaneously.

---

Asynchronicity does cut down on idling time, when it is awaitable.

Do note that asynchronous code can lead to time gains in cases where your synchronous code would otherwise be idling, e.g. waiting for a network response. This is not taken into account in the above example, which is exactly why I listed "cut [x]" jobs rather than "wait for [x] to boil".

Your job (the calculation) is not asynchronous code. It never idles (in a way that it's awaitable) and therefore it runs synchronously. This means you're not getting any benefit from running this asynchronously.

Reducing your code to a simpler example:

private static void CalculateStuff(Sample s, ProgressBar pb, ulong max)
{
    Thread.Sleep(5000);
}

Very simply put, this job takes 5 seconds and cannot be awaited. If you run 3 of these tasks at the same time, they will still be handled one after the other, taking 15 seconds total.

If the job inside your tasks were actually awaitable, you would see a time benefit. E.g.:

private static async void CalculateStuff(Sample s, ProgressBar pb, ulong max)
{
    await Task.Delay(5000);
}

This job takes 5 seconds but is awaitable. If you run 3 of these tasks at the same time, your thread will not waste time idling (i.e. waiting for the delay) and will instead start on the following task. Since it can await (i.e. do nothing for) these tasks at the same time, this means that the total processing time takes 5 seconds total (plus some negligible overhead cost).

---

According to Debug window, CPU usage increases with more tasks.

The managing of tasks takes a small overhead cost, which means that the total amounts of work (which can be measured in CPU usage over time) is slightly higher compared to synchronous code. That is to be expected.

This small cost usually pales in comparison to the benefits gained from well written asynchronous code. However, your code is simply not leveraging the actual benefits from asynchronicity, so you're only seeing the overhead cost and not its benefits, which is why your monitoring is giving you the opposite result of what you were expecting.

---

1. My computer has 2 processors, 10 cores each.

CPU cores, threads and tasks are three very different beasts.

Tasks are handled by threads, but they don't necessarily have a one-to-one mapping. Take the example of a team of 4 developers which has 10 bugs to resolve. While this means it's impossible for all 10 bugs to be resolved at the same time, these developers (threads) can take on the tickets (tasks) one after the other, taking on a new ticket (task) whenever they finished their previous ticket (task).

CPU cores are like workstations. It makes little sense to have less workstations (CPU cores) than you have developers (threads), since you'll end up with idling developers.

Additionally, you might not want your developers to be able to claim all workstations. Maybe HR and accounting (= other OS processes) also need to have some guaranteed workstations so they can do their job.
The company (= computer) doesn't just grind to a halt because the developers are fixing some bugs. This is what used to happen on single core machines - if one process claims the CPU, nothing else can happen. If that one process takes long or hangs, everything freezes.

This is why we have a thread pool. There is no straightforward real-world-analogy here (unless maybe a consultancy firm that dynamically adjusts how many developers it sends to your company), but the thread pool is basically able to decide how many developers are allowed to work at the company at the same time in order to ensure that development tasks can be seen to as fast as possible while also ensuring other departments can still do their work on the workstations as well.
It's a careful balancing act, not sending too many developers as that floods the systems, while also not sending too few developers as that means the work gets done too slowly.

The exact configuration of your threadpool is not something I can troubleshoot over a simple Q&A. But the behavior you describe is consistent with having less CPUs (dedicated to your runtime) and/or threads compared to how many tasks you have.

Answer 3

Unfortunately I can not give you a reason other than probably something with state machine magic that is happening but this significally increases performance:

private async void BThr4_Click(object sender, EventArgs e)
{
    Clear();
    DateTime start = DateTime.Now;

    await Task.Delay(100);
    Task<Sample> t1 = Task<Sample>.Run(() => CalculateStuff(PBar1, max / 4));
    Task<Sample> t2 = Task<Sample>.Run(() => CalculateStuff(PBar2, max / 4));
    Task<Sample> t3 = Task<Sample>.Run(() => CalculateStuff(PBar3, max / 4));
    Task<Sample> t4 = Task<Sample>.Run(() => CalculateStuff(PBar4, max / 4));

    Sample sample1 = await t1;
    Sample sample2 = await t2;
    Sample sample3 = await t3;
    Sample sample4 = await t4;

    TextBox.Text = (DateTime.Now - start).ToString(@"hh\:mm\:ss");

    t1.Dispose(); t2.Dispose(); t3.Dispose(); t4.Dispose();
}

// Calculate some math stuff
private static Sample CalculateStuff(ProgressBar pb, ulong max)
{
    Sample s = new Sample();
    ulong c = max / (ulong)pb.Maximum;

    for (ulong i = 1; i <= max; i++)
    {
        s.A = Math.Pow(s.A, 1.2);

        if (i % c == 0)
            pb.Invoke(new Action(() => pb.Value = (int)(i / c)));
    }

    return s;
}

This way you are not keeping Sample instances that the tasks have to access in the calling function but you create the instances within the task and then just return them to the caller after the task has completed.