How to parallelize a for loop in C++ creating the thread pool only once

Question

I have a C++ program that must run in Windows.

I have a game that has a main loop in the function WinMain which calls the Update functions in each iteration. I want to apply multithreading to a loop of update functions.

int __stdcall WinMain()
{
    // Windows initializations

    // Main loop
    {
        Game game = new Game();
        bool bExit = false;
        while (!bExit)
        {
            MSG msg;
            while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
            {
                if (msg.message == WM_QUIT) bExit = true;
                TranslateMessage(&msg);
                DispatchMessage(&msg);
            }

            game->Update();
        }
        delete game;
        game = nullptr;
    }

    // Windows destructions

    return 0;
}

// Game.cpp

void Game::Update() {
    // Loop that I want to be parallelized but I don't want to create the thread pool here because it's called on every frame
    for (size_t i = 0; i < entities.size(); i++) {
        entities[i]->Update();
    }
}

I tried using OpenMP but I couldn't place the #pragma omp parallel in the WinMain function (the app would crash) and if I put #pragma omp parallel for inside the Game::Update just before the loop it actually decreases performance because it's creating the thread loop in each frame.

I'm looking for a library-based or preferably a native solution that allows me to easily parallelize this loop.

The only requirements is that it works in Windows, I would prefer not using a library (like boost, although OpenMP is fine).

EDIT: I got it working with PPL's concurrency::parallel_for. It doesn't degrade performance but it doesn't increase it either... My question regarding this is: does this function create and destroy its thread pool on each call?

Answer 1

Here a solution with basic thread library. I simulated your Entity and Game classes.
Note that in this solution, worker thread are created and started ONE time at the beginning. They will be called on each Update call. When Update is not called, threads sleeps ...

I did my best to keep the architecture of your program. Note that we can use another implementation with std::thread, std::mutex, ... but I just wanted to give you an idea ...

#define NB_ENTITIES 10

class CEntity
{
public:
    void Update(){};
    ~CEntity () {}
};

typedef struct ThreadData
{
    HANDLE  hMutex;
    HANDLE  hMutexDestructor;
    CEntity *pCEntity;  
    DWORD   *dwStat;

} ThreadData;

//------------------------------------
// This function will call "Upadate"
//------------------------------------
DWORD WINAPI MyThreadFunction( LPVOID lpParam )
{
    ThreadData *ThreadDatpa      = (ThreadData*) lpParam;
    CEntity    *pEntity          = ThreadDatpa->pCEntity;
    HANDLE     hMutex            = ThreadDatpa->hMutex;
    HANDLE     hMutexDestructor  = ThreadDatpa->hMutexDestructor;
    DWORD      *dwStat           = ThreadDatpa->dwStat;

    while (true)
    {
        // When no update, thread sleep ... 0% CPU ...
        WaitForSingleObject(hMutex, INFINITE);
        if ( 0 == *dwStat ) break; // here thread stat for stopping 

        if ( nullptr != pEntity ) 
            pEntity->Update(); // Call your unpdate function ...
    }

    // Each worker thread must release it semaphore.
    // Destructor must get ALL released semaphore before deleting memory
    ReleaseSemaphore(hMutexDestructor, 1, NULL );

    return 0;
}

class Game 
{
public :
    vector<ThreadData*>  entities; // Vector of entities pointers
    vector<HANDLE>   thread_group; // vector of threads handle


    //This function must called ONE time at the beginning (at init)
    void StartThreads ()
    {
        DWORD  dwRet = 0;
        HANDLE hTemp = NULL;

        for (size_t i = 0; i <NB_ENTITIES; i++) 
        {
            CEntity *pCEntity = new CEntity (); // just to simulate entity

            // This semaphore is used to release thread when update is called
            HANDLE ghMutex= CreateSemaphore( NULL,0, 1, NULL); 

            // This semaphore is used when destruction to check if all threads is terminated
            HANDLE ghMutexDestructor= CreateSemaphore( NULL,0, 1, NULL);

            // create a new CEntity data ...
            ThreadData *pThreadData = new ThreadData ();

            pThreadData->pCEntity = pCEntity;
            pThreadData->hMutex   = ghMutex;
            pThreadData->hMutexDestructor = ghMutexDestructor;
            pThreadData->dwStat   = new DWORD (1); // default status = 1 

            entities.push_back ( pThreadData );         
        }

        // Here we start ONE time Threads worker.
        // Threads are stopped untile update was called
        for (size_t i = 0; i < entities.size(); i++) 
        {
            // Each thread has it own entity
            hTemp = CreateThread( NULL,0, MyThreadFunction, entities.at(i), 0, &dwRet);  
            if ( NULL != hTemp )
                thread_group.push_back (hTemp);
        }

    }

    // Your function update juste wakeup threads
    void Game::Update() {
        for (size_t i = 0; i < entities.size(); i++) 
        {
            HANDLE hMutex = entities.at(i)->hMutex;
            if ( NULL != hMutex )
                ReleaseSemaphore(hMutex, 1, NULL );
        }
    }


    ~Game()
    {
        // Modifie stat before releasing threads
        for (size_t i = 0; i < entities.size(); i++) 
            *(entities.at(i)->dwStat) = 0;

        // Release threads (status =0 so break ...)
        Update();

        // This can be replaced by waitformultipleobjects ...
        for (size_t i = 0; i < entities.size(); i++)
            WaitForSingleObject ( entities.at(i)->hMutexDestructor, INFINITE);

        // Now i'm sur that all threads are terminated
        for (size_t i = 0; i < entities.size(); i++)
        {
            delete entities.at(i)->pCEntity;
            delete entities.at(i)->dwStat;
            CloseHandle (entities.at(i)->hMutex);
            CloseHandle (entities.at(i)->hMutexDestructor);
            delete entities.at(i);
        }

    }
};

Answer 2

Providing a Boost.Asio example based on our discussion in the comments:

#include <thread>

#include <boost/asio.hpp>

namespace asio = boost::asio;

int main( int argc, int* argv[] ) {
    asio::io_context context{};

    asio::post( context, []() { /* any arbitrary job */ } );

    // this donates the new_thread to the thread pool
    std::thread new_thread{ [&](){ context.run(); } };

    // this donates the current thread to the thread pool
    context.run();
}

The main things to note here are:

1. asio::post allows you to submit arbitrary jobs to the io_context
2. calling io_context::run from a thread donates that thread to the thread pool that runs the context

If you want a pre-built thread pool, you can use a boost::asio::thread_pool, and call asio::post to put jobs on the thread pool in the same way.

You should only need to download Boost, you shouldn't need to install it. If you add BOOST_ERROR_CODE_HEADER_ONLY to you compilation, Boost.Asio is totally header-only.

EDIT:

You still need to run Boost's setup script, but you shouldn't need to build any of the libraries.