CODEWITHSUNDEEP
c++cmultithreadingwindows-server-2012-r2
Oleg Karasik
Oleg Karasik

Reputation: 969

Maximum number of User Mode Scheduler Threads

I am writing the UMS scheduler for managing large number of UMS worker threads and I have encountered a very strange issue. By some reason the maximum number of UMS threads that I can create is 8192 (there is no such problem with the normal threads, I can easily create >20 000 of them).

The error returned by the call to CreateRemoteThreadEx is 1450 "Insufficient system resources"

The PC has the following configuration:

I have tried the following:

  1. Increase Process Working Set (no effect)
  2. Increase number of SystemPages through registry (no effect, right now it is set to 0)
  3. I have checked the Free Pages, Non-paged and paged pools limits (non of them is starving during the CreateRemoteThreadEx call)
  4. I have tried that on another PC (2GB of RAM, Inter Core 2 Duo processors, Windows 8.1x64 and got the same magic number 8192).

Here is a code sample to test:

DWORD
UmsThreadRoutine(
    PVOID pState
)
{
    UNREFERENCED_PARAMETER(pState);
    return 0;
};

int main(int args, char **argv)
{
    PCHAR pMessage = "Program End";
    DWORD dwErrorCode = ERROR_SUCCESS;
    DWORD dwIteration = 0;
    PUMS_COMPLETION_LIST pUmsCompletionList = NULL;
    if (!::CreateUmsCompletionList(&pUmsCompletionList))
    {
        pMessage = "CreateUmsCompletionList";
        goto EXIT_POINT;
    }

    for (int i = 0; i < 20000; ++i)
    {
        dwIteration = i;
        LPPROC_THREAD_ATTRIBUTE_LIST pAttributes = NULL;
        SIZE_T szAttributes = 0;

        if (::InitializeProcThreadAttributeList(NULL, 1, 0, &szAttributes))
        {
            pMessage = "InitializeProcThreadAttributeList(Size)";
            goto EXIT_POINT;
        }

        pAttributes = (LPPROC_THREAD_ATTRIBUTE_LIST)::malloc(szAttributes);
        if (pAttributes == NULL)
        {
            pMessage = "malloc";
            goto EXIT_POINT;
        }
        if (!::InitializeProcThreadAttributeList(pAttributes, 1, 0, &szAttributes))
        {
            pMessage = "InitializeProcThreadAttributeList";
            goto EXIT_POINT;
        }

        PUMS_CONTEXT pUmsContext = NULL;
        if (!::CreateUmsThreadContext(&pUmsContext))
        {
            pMessage = "CreateUmsThreadContext";
            goto EXIT_POINT;
        }

        UMS_CREATE_THREAD_ATTRIBUTES umsCreationAttributes = { UMS_VERSION, pUmsContext, pUmsCompletionList };
        if (!::UpdateProcThreadAttribute(pAttributes, 0, PROC_THREAD_ATTRIBUTE_UMS_THREAD, &umsCreationAttributes, sizeof(UMS_CREATE_THREAD_ATTRIBUTES), NULL, NULL))
        {
            pMessage = "UpdateProcThreadAttribute";
            goto EXIT_POINT;
        }

        DWORD dwId = 0;
        HANDLE hThread = ::CreateRemoteThreadEx(
            ::GetCurrentProcess(),
            NULL,
            0,
            (LPTHREAD_START_ROUTINE)::UmsThreadRoutine,
            NULL,
            CREATE_SUSPENDED,
            pAttributes,
            &dwId
        );
        if (hThread == NULL)
        {
            pMessage = "CreateRemoteThreadEx";
            goto EXIT_POINT;
        }

        ::DeleteProcThreadAttributeList(pAttributes);
    }

    EXIT_POINT:
    dwErrorCode = ::GetLastError();
    ::printf("Program exited with Error Code '%d' on step '%s' on iteration '%d'", dwErrorCode, pMessage, dwIteration);
}

I run this snippet on another PC and it worked fine (I have created 20000 UMS Threads).

PC Configuration:

So basically this looks like a some sort of limitation (and looks like it can be changed).

Do you have any ideas?

Update: After installation of the latest updates on the box nothing has changed.

Upvotes: 1

Views: 978

Answers (1)

AlienRancher
AlienRancher

Reputation: 639

I can't find it right now but at some point I read that the UMS Kernel side implementation used Intel's LDT to map each UMS thread.

8192 is exactly the number of entries in the LDT table. The LDT could be managed in a number of creative ways in the Win7 Kernel so it could be that 8192 is the limit per process, per thread or maybe the limit per core. You'll have to experiment setting processor affinity or having more UMS scheduler threads to see if you can get past this limit.

Upvotes: 2

Related Questions