Threading Best Implementation

Question

I have an application where I have many "Searches" running at the same time (the searches take 1 - 10 sec to complete, depending on how many results the are available) The problem is that the delay when searching keeps getting bigger ( I Think because 25 Max threads) Im using Backgroundworker Class Atm. So I looked up a few other implementations:

Simple Examples :

    static void Main()
{

    for (int i = 0; i < 500; i++)
    {
        try
        {
            new Thread(new ParameterizedThreadStart(doWork)).Start(i);
        }
        catch { }
    }
    Console.ReadLine();
}
static void doWork(object i)
{
    Console.WriteLine(i + ": started");
    Thread.Sleep(1000);
    Console.WriteLine(i + " done");
    Thread.CurrentThread.Abort();
}

But I get exceptions that I abort the threads (wich worries me) So I tried whith the threadpool :

static void Main()
{

    for (int i = 0; i < 500; i++)
    {
        ThreadPool.QueueUserWorkItem(new WaitCallback(doWork), i);
    }
    Console.ReadLine();
}
static void doWork(object i)
{
    Console.WriteLine(i + ": started");
    Thread.Sleep(1000);
    Console.WriteLine(i + " done");
}

But this goes verry slow...

I'm still looking for the best implementation, Can anyone help me ?

EDIT: The DoWork Method Makes a Network Connection (and wait for it to complete) This is With An API so I can't do async

Answer 1

Thread pool tries to minimize number of threads created and instead of creating new threads for queued tasks it can wait for other threads in the pool to be freed. It does that for a reason - too many threads can hamper performance. But you can override minimum number of threads to be created before this throttling happens.

Here is your original code with correction to make it work fast:

static void Main()
{
    int minWorker, minIOC;
    ThreadPool.GetMinThreads(out minWorker, out minIOC);
    ThreadPool.SetMinThreads(50, minIOC);
    for (int i = 0; i < 500; i++)
    {
        ThreadPool.QueueUserWorkItem(new WaitCallback(doWork), i);
    }
    Console.ReadLine();
}

static void doWork(object i)
{
    Console.WriteLine(i + ": started");
    Thread.Sleep(1000);
    Console.WriteLine(i + " done");
}

Answer 2

Try this solution:

static void Main(string[] args)
{
    for (int i = 0; i < 100; i++)
    {
        Task t = new Task(doWork, i);
        t.Start();
    }
    Console.ReadLine();
}

static void doWork(object i)
{
    Console.WriteLine(i + ": started");
    Thread.SpinWait(20000000); // It depends on what doWork actually does whether SpinWait or Sleep is the most appropriate test
    //Thread.Sleep(1000);
    Console.WriteLine(i + " done");
}

With tasks you have a better way to control your work items and augment those with options which can improve performance. The default TaskScheduler for tasks uses ThreadPool to queue work items. (Read the bottom of this answer for further information about tasks.)

So, to answer the question we need to know what doWork actually does :-) But in general Task will be a good choice and a fine abstraction.

Parallel foreach

If you use a loop to spawn the jobs, and you're doing data parallelism then a parallel foreach can do the job:

Parallel.For(0, 500, i => doWork(i));

Links:

• http://msdn.microsoft.com/en-us/library/dd537609.aspx
• http://msdn.microsoft.com/en-us/library/dd460717.aspx

To a comment from spender

http://msdn.microsoft.com/en-us/library/dd537609.aspx

Tasks provide two primary benefits:

1) More efficient and more scalable use of system resources.

Behind the scenes, tasks are queued to the ThreadPool, which has been enhanced with algorithms (like hill-climbing) that determine and adjust to the number of threads that maximizes throughput. This makes tasks relatively lightweight, and you can create many of them to enable fine-grained parallelism. To complement this, widely-known work-stealing algorithms are employed to provide load-balancing.

2) More programmatic control than is possible with a thread or work item.

Tasks and the framework built around them provide a rich set of APIs that support waiting, cancellation, continuations, robust exception handling, detailed status, custom scheduling, and more.

Updated answer

Well, it's unfortunately a bad API because it doesn't allow you to do it asynchronously. It probably is going slow because you start so many connection at the same time (or you start too few).

Try this:

var jobs = new[] { 1, 2, 3};
var options = new ParallelOptions { MaxDegreeOfParallelism = 3 };
Parallel.ForEach(jobs, options, i => doWork(i));

And experiment with the value of MaxDegreeOfParallelism.

Answer 3

Anything that involves queueing 500 items up for the ThreadPool to process will not run with optimal throughput. The ThreadPool is generally quite reluctant to spin up extra threads as this kind of usage is not what the designers intended.

It sounds to me like you're IO bound, in which case you could perform the IO asynchronously and service everything with very few threads. However, without knowing more about your workload, it's a bit of a guessing game.

Answer 4

If search is CPU bound, I'd use Parallel.For or parallel linq, with a manually specified MaxDegreeOfParallelism. Typically the number of virtual cores is the optimal number of threads in that case.

If the search waiting on something external(for example IO bound, waiting for responses over the network,...), I'd look into non blocking APIs, so you don't need a thread per search.

Answer 5

1) If you call to abort, you get an ThreadAbortException: Thread.Abort

2) ¿500 threads? I think you are doing something in the bad way.(unless you are working with GPUs)

Answer 6

Don't use Thread.Abort. That's a rather violent end to a poor working threat that just wanted to end gracefully. You can just let the original doWork method end, and the thread will be released.

And regarding your second solution - you're queuing up 500 threads in a second, which is a lot more than the ThreadPool can run concurrently. It takes more time because they're running one after the other.

Another option, in .NET 4.0, is the Task library in System.Threading.Tasks, which is a smarter threadpool-based solution you might consider.

But to get back to your original problem - what exactly do you mean by "the delay gets longer" when using BackgroundWorkers? Do you mean that each individual search takes longer (closer to 10 seconds) when there are multiple searches going on? If so, I'd try to look for the bottleneck somewhere else. What's a "search"? Are you accessing a database? A network connection? Perhaps you have locks in that part of the application that are causing bottlenecks.