CODEWITHSUNDEEP
cpthreads
Martin Vegter
Martin Vegter

Reputation: 527

c pthreads: run at most 4 threads concurrently at any time

I have a program in C, which takes arbitrary number of files as commandline argument, and calculates sha1sum for every file. I am using pthreads, so that I can take advantage of all 4 my cores.

Currently, my code runs all threads in parallel at the same time. Here is a snippet:

c = 0;
for (n = optind; n < argc; n++) {
    if (pthread_create(&t[c], NULL, &sha1sum, (void *) argv[n])) {
        fprintf(stderr, "Error creating thread\n");
        return 1;
    }
    c++;
}

c = 0;
for (n = optind; n < argc; n++) {
    pthread_join(t[c], NULL);
    c++;
}

Obviously, it is not efficient (or scalable) to start all threads at once.

What would be the best way to make sure, that only 4 threads are running at any time? Somehow I need to start 4 threads at the beginning, and then "replace" a thread with new one as soon as it completes.

How can I do that ?

Upvotes: 1

Views: 468

Answers (2)

P.P
P.P

Reputation: 121357

Obviously, it is not efficient (or scalable) to start all threads at once.

Creating 4 threads is not necessarily provides the best performance on a 4 core machine. If the threads are doing IO or waiting on something, then creating more than 4 threads could also result in better performance/efficiency. You just need to figure out an approximate number based on the work your threads do and possbily a mini-benchmark.

Regardless of what number you choose (i.e. number of threads), what you are looking for is a thread pool. The idea is to create a fixed number of threads and feed them work as soon as they complete.

See C: What's the way to make a poolthread with pthreads? for a simple skeleton. This repo also shows a self-contained example (check the license if you are going to use it). You can find many similar examples online.

Upvotes: 2

Scott Hunter
Scott Hunter

Reputation: 49803

The thing you are looking for is a semaphore; it will allow you to restrict only 4 threads to be running at a time. You would/could start them all up initially, and it will take care of letting a new one proceed when a running one finishes.

Upvotes: 0

Related Questions