CODEWITHSUNDEEP
cparallel-processingopenmp
JuMoGar
JuMoGar

Reputation: 1760

How to parallelize for loop inside other for loop with OpenMP

I want obtain best perfomance parallelizing this loops:

// EXAMPLE
for (;;) {
    // DO SOMETHING
    for(;;) {
        // DO SOMETHING
    }
}

I know three ways of parallelize about example:

// EXAMPLE - FIRST LOOP PARALLEL
#pragma omp parallel for
for (;;) {
    // DO SOMETHING
    for(;;) {
        // DO SOMETHING
    }
}

// EXAMPLE - FIRST AND SECOND LOOP PARALLEL NO NESTED
omp_set_nested(0); // default option
#pragma omp parallel for
for (;;) {
    // DO SOMETHING
    #pragma omp parallel for
    for(;;) {
        // DO SOMETHING
    }
}

// EXAMPLE - FIRST AND SECOND LOOP PARALLEL NESTED
omp_set_nested(1);
#pragma omp parallel for
for (;;) {
    // DO SOMETHING
    #pragma omp parallel for
    for(;;) {
        // DO SOMETHING
    }
}

Which is the best way of doing that? Or in which situations should I use one or other?

Thank you.

Upvotes: 0

Views: 1502

Answers (1)

Brice
Brice

Reputation: 1580

A single level of parallelization is almost always the best, i.e. Example 1: first loop parallel.

Example 2 will in fact do the same as example 1 (the second #pragma omp parallel for statement is ignored as nested parallelization is disabled).

Example 3 is likely a bad idea. The overhead for spawning new threads is incurred at each iteration from loop 1. Cases where it would beat parallelizing the first loop only (or second loop only) are non trivial. It is not guaranteed to be supported by all compilers (some may still ignore the nested part).

Example 4 is another option, when parallelization of the outermost loop can't be done for any reason (memory use, for instance) but we want to spawn the threads only once and re-use them at each iteration of the outermost loop. Some synchronization overhead will be incurred at each iteration, but much less than what repeatedly spawning threads would cost.

// EXAMPLE - SECOND LOOP PARALLEL
#pragma omp parallel
for (;;) {
    // DO SOMETHING EVERY THREAD SHOULD DO
    // e.g. declare local variables, increment private counter...
    #pragma omp single
    {
    // DO SOMETHING ONCE ONLY
    // e.g. read data from a file, initialize a shared variable
    }
    #pragma omp for
    for(;;) {
        // DO SOMETHING
    }
}

Upvotes: 2

Related Questions