How to limit number of tasks in C# so that multiple works can be done through that number of tasks

Question

I want to run 3 tasks parallel which will take input form one array of size n in a for loop. Once one task is completed it will take another input from the array and so on.

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            int []Arr ={1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
            List<Task> tasks = new List<Task>();
            int maxConcurrency = 5;
            using (SemaphoreSlim concurrencySemaphore = new SemaphoreSlim(maxConcurrency))
            {
                foreach (var x in Arr)
                {
                    concurrencySemaphore.Wait();
                    var t = Task.Run(async() =>
                    {
                        try
                        {
                            await concurrencySemaphore.WaitAsync();
                            print(x);
                        }
                        finally
                        {
                            concurrencySemaphore.Release();
                        }
                    });
                    int number = Process.GetCurrentProcess().Threads.Count;
                    Console.WriteLine("Thread {0} count {1}", t.Id, number);
                    tasks.Add(t);
                }
                Task.WaitAll(tasks.ToArray());
            }
            Console.ReadLine();
        }
        static void print(int x)
        {
            Console.WriteLine("Hello World! {0}", x);
        }
    }
}


    I expect the output should be hello world! 1-10 but the actual output is
    Hello World! 1
    Thread 2 count 12
    Hello World! 2
    Thread 4 count 12
    Hello World! 3
    Thread 6 count 12
    Hello World! 4
    Thread 8 count 12
    Thread 10 count 12

If you have any other suggestion then you are welcome.

Answer 1

The problem with your code is that you have two wait statements, one blocking and one non-blocking.

concurrencySemaphore.Wait();
await concurrencySemaphore.WaitAsync();

You only need to wait once for an available semaphore. So If you comment out the concurrencySemaphore.Wait() line, your code will work as expected.

Other things to note about your code:

Since you only designated one method for asynchronous processing (the WaitAsync method), this code will run exactly the same if you replace WaitAsync with Wait. So, when using WaitAsync in this situation, the task will first execute the WaitAsync method, if it cannot enter a semaphore, it will put that statement on hold and execute another async statement. But since there are no more async methods defined, the code will just end up blocking until it can enter a semaphore.

To prove your code is processing 5 Tasks at a time, you should add a call to the Sleep method in your PRINT function. This will allow you to visually see the number of Tasks being process at any given time.

Example:

static void print(int x)
{
    Console.WriteLine("Hello World! {0}", x);
    Thread.Sleep(3000);  //Wait 3 seconds to allow other Task to process before exiting
}

To demonstrate here is a tweak version of your code with the Async/Await statements replace with Wait.

class Program
{
    static void Main(string[] args)
    {
        int[] Arr = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
        int maxConcurrency = 5;
        List<Task> tasks = new List<Task>();

        using (SemaphoreSlim concurrencySemaphore = new SemaphoreSlim(0, maxConcurrency))
        {
            // Create a Task object for each element in the array
            foreach (var x in Arr)
            {

                tasks.Add(Task.Run(() =>
                {
                    // Block until the task can enter a semaphore
                    concurrencySemaphore.Wait();

                    // Do some work
                    print(x);

                    // Signal you are done with the semaphore
                    concurrencySemaphore.Release();
                }
                ));
        }

        // Wait a haft a second to allow all tasks to start and block.
        Thread.Sleep(500);

        // This will release "maxConcurrency" tasks to be process at a given time.
        concurrencySemaphore.Release(maxConcurrency);

        // Main thread waits for all tasks to complete.
        Task.WaitAll(tasks.ToArray());
    }

    Console.WriteLine("Press any key to exit program...."); Console.ReadKey();
    } /* end Main */

    static void print(int x)
    {
        Console.WriteLine("Hello World! {0}", x);
        Thread.Sleep(3000);  //Wait 3 seconds to allow other Task to process before exiting
    }
} /* end class */

The results will be similar to what is shown below:

Hello World! 2
Hello World! 4
Hello World! 5
Hello World! 1
Hello World! 6

-- A 3 second pause --

Hello World! 8
Hello World! 9
Hello World! 3
Hello World! 10
Hello World! 7

-- A 3 second pause --

Press any key to exit program....

-- Update --

As mention by Nick, you can also use the Parallel.ForEach method to accomplish the same thing.

Example:

Parallel.ForEach (Arr,
    new ParallelOptions { MaxDegreeOfParallelism = maxConcurrency },
    (x) => { print(x); } );

Answer 2

Alternatively, you can use Parallel class and specify MaxDegreeOfParallelism. However, just like the documentation says, normally you will not want to do that. See the remarks in the second link.