Parallel threads and control?

Question

I have a function like this:

private void GetRSS(int start, int end)
{
    for (int i = start; i < end; i++)
    {
        string content = string.Empty;
        using (WebClient client = new WebClient())
        {
            //some code here get html content
        }
        // some code here parse content
    }
}

In order to minimize the amount of time running to get all the needed data, I would like to run the function 4 times with different ranges at the same time and then merge the results or use a thread safe list or dictionary.

My question is, how could I run this function in 4 separated threads and still be able to control if one of the threads still working or not to know when it ends ?

My first idea was to declare each thread:

private Thread _controler;
private Thread _worker1;
private Thread _worker2;
private Thread _worker3;
private Thread _worker4;
private bool _isRunning = false;

Then I would start the controler and from inside the controler I would call each thread to execute the function above and keep track of each thread from the controler, something like:

private void _ControlerStart()
{
    _worker1 = new Thread(GetRSS);
    try
    {
        _worker1.Start(1, 7711);
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
    // repeat the above to all the 4 threads
    _isRunning = true;
    while (_isRunning)
    {
        if (_worker1.ThreadState != ThreadState.Running && _worker2.ThreadState != ThreadState.Running && _worker3.ThreadState != ThreadState.Running && _worker4.ThreadState != ThreadState.Running)
            _isRunning = false;
    }
    MessageBox.Show("Done");
}

While thinking on all this mess above I realize d that this is not the best way to do what I wanted and here I am ;).

How can I manage more than 1 thread to run the same function and yet be able to know when each thread has ended working to close or save or merge the data or do whatever else I have left to do from a main thread?

Answer 1

Is there a specific reason not to use TPL? http://msdn.microsoft.com/en-us/library/dd460717.aspx

As you might have noticed, the problem with supplied code is that it's not declarative. You are telling the machine how to do it, and not what you want done.

EDIT In my comment, I suggested using the following code.

In OP's comment, I saw

I have 30k pages to retrieve where I have 10 records on each page so I want to split the 30k pages along 4 threads where each page needs to be parsed after downloaded into a string

The first part defines the action to be performed, and the second part clearly describes the hand-holding for the PC. What I am proposing is to stop holding PC's hand, and think in terms of high-level actions to be performed.

1. Ask for N pages
2. When any one page is available, consume the result
3. When all N pages are available and consumed, collate that result

This calls for async programming. In .NET 4.0, it's mainly available under F# 2.0. C# 4.0 can use APM with continuations, but I would not recommend it.

C# vNext has support for this, see VS 2012.

Answer 2

In task way, you can use TaskFactory.ContinueWhenAll Method to execute some action when all tasks has complited.

    private void ButtonOnClick(object sender, RoutedEventArgs routedEventArgs) {
        const int n = 15;
        var tasks = new Task<int>[n];
        for (int i = 0; i < n; i++) {
            tasks[i] = Task.Factory.StartNew(
                () => {
                    Thread.Sleep(500);
                    return 100;
                });
        }

        Task.Factory.ContinueWhenAll(
            tasks,
            ts => { text.Text = string.Format("Sum: {0}", ts.Sum(task => task.Result)); },
            CancellationToken.None,
            TaskContinuationOptions.None,
            TaskScheduler.FromCurrentSynchronizationContext());
    }

Here I am using TaskScheduler.FromCurrentSynchronizationContext() in order to execute continuation task in UI thread.

Answer 3

private static void GetRSS(int start, int end)
{
    var bunchSize = (end - start) / 4 + 1;

    var threads = new List<Thread>();

    for (int i = 0; i < 4; i++)
    {
        var currStart = start + i * bunchSize;
        var currEnd = currStart + bunchSize;

        if (currEnd > end)
        {
            currEnd = end;
        }

        var thread = new Thread(() =>
                                    {
                                        // thread logic using currStart and currEnd

                                        string content = string.Empty;
                                        using (WebClient client = new WebClient())
                                        {
                                            //some code here get html content
                                        }
                                        // some code here parse content
                                    });

        threads.Add(thread);
        thread.Start();
    }

    foreach (var thread in threads)
    {
        thread.Join();
    }
}