CODEWITHSUNDEEP
c++multithreadingwinapi
Jake
Jake

Reputation: 2907

Increasing MAXIMUM_WAIT_OBJECTS for WaitforMultipleObjects

What is the simplest way to wait for more objects than MAXIMUM_WAIT_OBJECTS? MSDN lists this:

But neither are them are very clear. The situation would be waiting for an array of over a thousand handles to threads.

Upvotes: 5

Views: 7226

Answers (4)

Jan Ringoš
Jan Ringoš

Reputation: 141

Since Windows 8 there's a third way: You can use NT API to get IOCP completion packet when an event, semaphore or a thread handle gets signalled (thread finishes).

To do that you create this completion packet with NtCreateWaitCompletionPacket , and bind all three handles (this packet, IOCP and thread/event) together through poorly documented (but intuitive enough) NtAssociateWaitCompletionPacket. When you receive the completion, you can call it again to reestablish the association and receive the next one, when the object is signalled again.

See full example here: https://github.com/tringi/win32-iocp-events

Upvotes: 0

Mike Robinson
Mike Robinson

Reputation: 31

I encountered this limitation in WaitForMultipleObjects myself and came to the conclusion I had three alternatives:

  • OPTION 1. Change the code to create separate threads to invoke WaitForMultipleObjects in batches less than MAXIMUM_WAIT_OBJECTS. I decided against this option, because if there are already 64+ threads fighting for the same resource, I wanted to avoid creating yet more threads if possible.
  • OPTION 2. Re-implement the code using a different technique (IOCP, for example). I decided against this too because the codebase I am working on is tried, tested and stable. Also, I have better things to do!
  • OPTION 3. Implement a function that splits the objects into batches less than MAXIMUM_WAIT_OBJECTS, and call WaitForMultipleObjects repeatedly in the same thread.

So, having chosen option 3 - here is the code I ended up implementing ...

class CtntThread
{
    public: 
        static DWORD WaitForMultipleObjects( DWORD, const HANDLE*, DWORD millisecs );
};

DWORD CtntThread::WaitForMultipleObjects( DWORD count, const HANDLE *pHandles, DWORD millisecs )
{
    DWORD retval = WAIT_TIMEOUT;

    // Check if objects need to be split up. In theory, the maximum is
    // MAXIMUM_WAIT_OBJECTS, but I found this code performs slightly faster
    // if the object are broken down in batches smaller than this.
    if ( count > 25 )
    {
        // loop continuously if infinite timeout specified
        do
        {
            // divide the batch of handles in two halves ...
            DWORD split = count / 2;
            DWORD wait = ( millisecs == INFINITE ? 2000 : millisecs ) / 2;
            int random = rand( );

            // ... and recurse down both branches in pseudo random order
            for ( short branch = 0; branch < 2 && retval == WAIT_TIMEOUT; branch++ )
            {
                if ( random%2 == branch ) 
                {
                    // recurse the lower half
                    retval = CtntThread::WaitForMultipleObjects( split, pHandles, wait );
                }
                else
                {
                    // recurse the upper half
                    retval = CtntThread::WaitForMultipleObjects( count-split, pHandles+split, wait );
                    if ( retval >= WAIT_OBJECT_0 && retval < WAIT_OBJECT_0+split ) retval += split;
                }
            }
        }
        while ( millisecs == INFINITE && retval == WAIT_TIMEOUT );
    }
    else
    {
        // call the native win32 interface
        retval = ::WaitForMultipleObjects( count, pHandles, FALSE, millisecs );
    }

    // done
    return ( retval );
}

Upvotes: 3

COrthbandt
COrthbandt

Reputation: 191

If you find yourself waiting on tons of objects you might want to look into IO Completion Ports instead. For large numbers of parallel operations IOCP is much more efficient.

And the name IOCP is misleading, you can easily use IOCP for your own synchronization structures as well.

Upvotes: 8

Jon Cage
Jon Cage

Reputation: 37448

Have a look here.

If you need to wait on more than MAXIMUM_WAIT_OBJECTS handles, you can either create a separate thread to wait on MAXIMUM_WAIT_OBJECTS and then do a wait on these threads to finish. Using this method you can create MAXIMUM_WAIT_OBJECTS threads each of those can wait for MAXIMUM_WAIT_OBJECTS object handles.

Upvotes: 1

Related Questions