c++ Handling multiple threads in a main thread

Question

I am a bit new to multi threading, so forgive me if these questions are too trivial.

My application needs to create multiple threads in a thread and perform actions from each thread.

For example, I have a set of files to read, say 50 and I create a thread to read these files using CreateThread() function.

Now this main thread creates 4 threads to access the file. 1st thread is given file 1, second file 2 and so on.

After 1st thread completed reading file 1 and gives main thread the required data, main thread needs to invoke it with file 5 and obtain data from it. Similar goes for all other threads until all 50 files are read.

After that, each thread is destroyed and finally my main thread is destroyed.

The issue I am facing is:

1) How to stop a thread to exit after file reading?

2) How to invoke the thread again with other file name?

3) How would my child thread give information to main thread?

4) After a thread completes reading the file and returns the main thread a data, how main thread would know which thread has provided the data?

Thanks

Answer 1

This is a very common problem in multi-threaded programming. You can view this as a producer-consumer problem: the main thread "produces" tasks which are "consumed" by the worker threads (s. e.g. http://www.mario-konrad.ch/blog/programming/multithread/tutorial-06.html) . You might also want to read about "thread pools".

I would highly recommend to read into boost's Synchronization (http://www.boost.org/doc/libs/1_50_0/doc/html/thread.html) and use boost's threading functionality as it is platform independent and good to use.

To be more specific to your question: You should create a queue with operations to be done (usually it's the same queue for all worker threads. If you really want to ensure thread 1 is performing task 1, 5, 9 ... you might want to have one queue per worker thread). Access to this queue must be synchronized by a mutex, waiting threads can be notified by condition_variables when new data is added to the mutex.

1.) don't exit the thread function but wait until a condition is fired and then restart using a while ([exit condition not true]) loop

2.) see 1.

3.) through any variable to which both have access and which is secured by a mutex (e.g. a result queue)

4.) by adding this information as the result written to the result queue.

Another advice: It's always hard to get multi-threading correct. So try to be as careful as possible and write tests to detect deadlocks and race conditions.

Answer 2

Whether you use a thread pool or not to execute your synchronies file reads, it boils down to a chain of functions or groups of functions that have to run serialized. So let's assume, you find a way to execute functions in parallel (be it be starting one thread per function or by using a thread pool), to wait for the first 4 files to read, you can use a queue, where the reading threads push there results into, the fifth function now pulls 4 results out of the queue (the queue blocks when empty) and processes. If there are more dependencies between functions, you can add more queues between them. Sketch:

void read_file( const std::string& name, queue& q )
{
    file_content f= .... // read file
    q.push( f )
}

void process4files( queue& q )
{
    std::vector< file_content > result;
    for ( int i = 0; i != 4; ++i )
        result.push_back( q.pop() ); 

    // now 4 files are read ...
    assert( result.size() == 4u );
}

queue       q;
thread t1( &read_file, "file1", q );
thread t2( &read_file, "file2", q );
thread t3( &read_file, "file3", q );
thread t4( &read_file, "file4", q );
thread t5( &process4files, q );

t5.join();

I hope you get the idea.

Torsten

Answer 3

The typical solution for this kind of problem is using a thread pool and a queue. The main thread pushes all files/filenames to a queue, then starts a thread pool, ie different threads, in which each thread takes an item from the queue and processes it. When one item is processed, it goes on to the next one (if by then the queue is not yet empty). The main thread knows everything is processed when the queue is empty and all threads have exited.

So, 1) and 2) are somewhat conflicting: you don't stop the thread and invoke it again, it just keeps running as long as it finds items on the queue. For 3) you can again use a queue in which the thread puts information, and from which the main thread reads. For 4) you could give each thread an id and put that together with the data. However normally the main thread should not need to know which thread exactly processed data.

Some very basic pseudocode to give you an idea, locking for threadsafety omitted:

//main
for( all filenames )
  queue.push_back( filename );

//start some thread
threadPool.StartThreads( 4, CreateThread( queue ) );

//wait for threads to end
threadPool.Join();

//thread
class Thread
{
public:
  Thread( queue q ) : q( q ) {}

  void Start();

  bool Join();

  void ThreadFun()
  {
    auto nextQueueItem = q.pop_back();
    if( !nextQueuItem )
      return; //q empty
    ProcessItem( nextQueueItem );
  }
}