CODEWITHSUNDEEP
c#.netmultithreading
Hand-E-Food
Hand-E-Food

Reputation: 12814

Run one method in multiple threads as decided by the system

I have a method that processeswords in two lists, a priority list and a standard list.

ConcurrentBag<Word> PriorityWords = ...;
ConcurrentBag<Word> UnprocessedWords = ...;

public void ProcessAllWords()
{
    while (true)
    {
        Word word = SelectWordToProcess();
        if (word == null) break;
        ProcessWord(word);
    }
}

private Word SelectWordToProcess()
{
    Word word;
    if (PriorityWords.TryTake(out word) || UnprocessedWords.TryTake(out word))
        return word;
    else
        return null;
}

public void ProcessWord(Word word) { ... }

I want to run this method on multiple cores. Currently, I simply open one thread per processor:

for (int i = 0; i < Environment.ProcessorCount; i++)
{
    new Thread(ProcessAllWords).Start();
}

Is there a more suitable way that lets the system decide how many threads to open based on current system performance, similar to Parallel.ForEach()?

EDIT: More detail on the application.

The word list is prepopulated with ~180,000 words and every word is to be permutated with every other word. ProcessAllWords is an O(n²) operation. The threads will all run flat-out until all words are processed, then terminate. While the threads are running, I can asynchronously give priority to specific words by adding them to the PriorityWords list. Initial tests show my system processes about 5 words/sec, so that's 10 hours of 100% CPU processing.

Upvotes: 0

Views: 280

Answers (1)

John Tseng
John Tseng

Reputation: 6352

Your method of starting Environment.ProcessorCount threads is good. The Task Parallel Library will do the automatic scheduling that you're looking for, but at the cost of over subscribing your CPU. This will decrease the responsiveness of your application to priority words.

As for the various methods of using TPL, parallel for and task factory will both queue up a bunch of words, making it very unresponsive to priority. You can maintain your priority with a generator method and PLINQ, but then you get a fixed number of threads as you have now. You can either set your thread count or use the default of 2xEnvironment.ProcessorCount. All in all, since your task is CPU bound, I would keep your current implementation.

Upvotes: 1

Related Questions