CODEWITHSUNDEEP
clinuxpthreadsposix
NickSoft
NickSoft

Reputation: 3335

Limit the amount of concurrent threads (with pthreads)

I have to write a C application that posts info on web using CURL. The application must run up to N (let's say 10) requests in parallel at most. How can I wait for ANY thread to finish, rather than specific thread using pthread_join().

I read about pthread_cond_wait, but most examples are how the control thread (main) wakes up a worker thread. I need just the opposite - worker threads must be able to signal/wakeup parent thread before exiting.

Update: Actually I need a way to make manager thread sleep and when a worker thread finishes it's job it should wake the manager thread to give it another job. It doesn't matter if the thread will end and new thread will be created for the job or thread pool will be used. Threre still needs to be a way to signal the manager that a job is finished.

I hope that I DON'T get this suggestion:

while(asleep){
  for(i = 0; i< threadCount; i++){
    pthread_mutex_lock(mutex);
    if(threads[i] == IDLE_STATE)
      startNewJob();
    pthread_mutex_unlock(mutex);
    usleep(100*1000);
  }
}

Upvotes: 4

Views: 2526

Answers (3)

caf
caf

Reputation: 239011

Condition variable are what you're after. They can be used just as easily to signal the manager thread from the worker thread as vice-versa.

Your strawman example at the end is actually very similar to what you could do using a condition variable:

pthread_mutex_lock(&mutex);
while (!finished) {
  for(i = 0; i < threadCount; i++) {
    if(threads[i] == IDLE_STATE)
      startNewJob(i);
  }

  /* Not finished, and no idle threads, so wait */
  if (!finished)
    pthread_cond_wait(&cond, &mutex);
}
pthread_mutex_unlock(&mutex);

When a thread is done, it would simply do:

pthread_mutex_lock(&mutex);
threads[self] = IDLE_STATE;
pthread_cond_signal(&cond);
pthread_mutex_unlock(&mutex);

Upvotes: 1

Douglas Leeder
Douglas Leeder

Reputation: 53310

You want a condition variable, the same functions you have been looking at, but just turned around. The condition you are waiting for is "a worker thread has finished some work".

the main thread does:

  • lock mutex
  • while no thread has finished: pthread_cond_wait
  • unlock mutex
  • schedule work
  • loop

Each worker thread, when it finishes work does:

  • lock mutex
  • mark completed
  • pthread_cond_signal
  • unlock mutex
  • exit or wait for work

Upvotes: 1

simonc
simonc

Reputation: 42165

Rather than creating/destroying threads on demand, it'll be easier to create a pool of 10 worker threads on startup and have your main program feed jobs to them.

On startup, you'd create an array of 10 workers. These might look something like

typedef struct worker
{
    pthread_t       thread;
    pthread_cond_t  cond;
    pthread_mutex_t mutex;
    struct job*     job;
    int             quit;
} worker;

The manager would delegate jobs to each thread in turn by setting their job member then signalling the worker.

Each worker would loop until quit was non-zero, waiting on its condition being signalled. After each signal it would read/process its job before reporting the results then waiting on its condition again.

Edit: You're not keen on thread pools. You could instead try giving each thread a unique id; store some mapping between ids and other properties of each thread in the manager. When each thread completes, have it add its id to a list owned by the manager then signal a condition in the manager. Each time the manager wakes, it can pull the head from the list, lookup the appropriate thread, read back its job results then join the thread. Note that the manager's list here will be accessed by multiple threads so reads/writes will need to be protected by a mutex.

Edit2: You'd like to know more about conditions and don't find the examples you've found helpful. I'm not sure this'll be any better but here's some code I've written. The OsSemaphore* functions wrap conditions into a simple Wait/Signal API.

Upvotes: 4

Related Questions