Threadpool + Polling C# .Net 3.5

Question

Hi I am new to multithreading and would like to ask for your advice and guidance.

We have a service running on our server to poll data for notifications on our clients. We wanted the service to process data faster. Currently, our existing service polls and processes data on a single thread which sometimes causes delay to the notifications on hourly bases. My plan is to use ThreadPool to process data concurrently. I have this piece of code that simulates my plan and idea.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Configuration;
using System.Data;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Mail;
using System.Security;
using System.Text;
using System.Threading;
using System.Xml;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using System.Net.Sockets;
using System.Security.Authentication;
using System.Web; 

namespace ThreadPooling
{
    class Program
    {
        static int nMaxRecord = 0;
        static ManualResetEvent mre = new ManualResetEvent(false);
        static Timer TestThread = null;
        static void Main(string[] args)
        {
            TestThread = new Timer(new TimerCallback(ProcessWithThreadPoolMethod), null, 500, Timeout.Infinite);
            Thread.Sleep(Timeout.Infinite);
        }

        static void ProcessWithThreadPoolMethod(object ostate) // Sample processing of data
        {
            nMaxRecord = 1300;
            ThreadPool.SetMaxThreads(3, 0);
            for (int i = 0; i < 1300; i++)
            {
                ThreadPool.QueueUserWorkItem(ProcessWithThreadMethod, i);
            }

            mre.WaitOne();
            Console.WriteLine("Test");

            TestThread.Change(5000, Timeout.Infinite);
        }

        static void ProcessWithThreadMethod(object callback)
        {
            for (int i = 0; i <= 10; i++)
            {
                Console.WriteLine((int)callback);
            }

            if(Interlocked.Decrement(ref nMaxRecord) == 0)
            {
                mre.Set();
            }
        }
    }
}

While running the console application, I noticed that the thread count keeps increasing although I limited the maxthreads in the ThreadPool by 3. Am I doing the right thing? Would like to ask some guidance and Pros and Cons on my concept.

Answer 1

You cannot throttle the threadpool, but why not just have a simple increment/decrement counter that you check before launching a new thread ?

In pseudo code-

volatile int currentThreadCount = 0;

void myWorkLauncher()
{
   while(<I have work to do>)
   {
      if(currentThreadCount < threshold)
      {
          currentThreadCount ++;
          ThreadPool.QueueUserWorkItem(workerFunc);
      }
      else
      {
          Thread.Sleep(500);
      }
   }
   Thread.Sleep(500);
}

The last line of the workedFunc just decrements the value.

You can do all sorts of fancy stuff like wrapping your workerFunc in an Action() which itself decrements the counter, preventing your workerFunc from needing any connection to the myWorkLauncher class. Alternatively you can replace the simplistic Thread.Sleep with a AutoResetEvent or similar.

Answer 2

You should test the return value of:

ThreadPool.SetMaxThreads(3, 0); //returns false on normal machines.

It's unable to process the change due to this:

You cannot set the maximum number of worker threads or I/O completion threads to a number smaller than the number of processors on the computer. To determine how many processors are present, retrieve the value of the Environment.ProcessorCount property. In addition, you cannot set the maximum number of worker threads or I/O completion threads to a number smaller than the corresponding minimum number of worker threads or I/O completion threads. To determine the minimum thread pool size, call the GetMinThreads method.

See: MSDN

---

So, what you can do is something like this;

ThreadPool.SetMaxThreads(16, 16);

But I assume you were trying to down-throttle the ThreadPool. In general that's not a good idea. You'll need an alternative for such logic.

A semaphore might be an option, as described here, or the pattern @Fildor describes.