CODEWITHSUNDEEP
cwinapimemory-managementheap-memory
user3601163
user3601163

Reputation:

Allocating all the memory makes my system crash

I have detected a bug in my program during the load test and investigated. At first glance, I thought there were an error in my code. However it works with a smaller amount of data and it executed as expected debugging step by step. So I reduced my code to the following sample. There is no processing, only allocation:

void main(void) {
    const signed int n = 100000000; /* high on purpose */
    signed int k;
    char **Buffer = NULL;

    Buffer = (char**) HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, n * sizeof(char*));
    if (!Buffer) {
        printf("Fail to allocate the big buffer\n");
        return;
    }
    for (k = 0; k < n; ++k) {
        Buffer[k] = (char*) HeapAlloc(GetProcessHeap(), 0, 3 * sizeof(char)); /* using malloc also causes failure */
        if (Buffer[k] == NULL) {
            printf("Out of memory on buffer no. %d\n", k);
            /* break; */
        }
    }       
    /* break point here */
    for (--k; k >= 0; --k)
        if (Buffer[k])
            HeapFree(GetProcessHeap(), 0, Buffer[k]);
    HeapFree(GetProcessHeap(), 0, Buffer);
}

The computer has 12Go of RAM and no page file (swap). When I run this code, the memory consumption increases as expected to the point where I have no more memory available. The allocations tend to be slower at the end, then start to fail. After, my program and other applications crash, including bits of Windows (7 SP1 64bit) and my debugger (Visual Studio 2013). I cannot use the break statement because the code is to be put it in an OpenMP block (I will use a flag). Still, I tested and it does not change anything.

Any ideas why it makes everything crash?

Here is my guess. As I allocate a high number of instances of a small amount of memory, I probably reach the case where there is zero free page left (ore very close to). So other applications cannot allocate memory neither, even a small amount of it. They might either not handle the lack of memory well, either require it absolutely and crash. Visual Studio or Windows might be concerned and crashing my application when crashing themselves. Once I got a BSD with the PAGE_FAULT_IN_NONPAGED_AREA error.

Is my guess right on the cause that leads to the crash or am I missing something? Is my code wrong or is there a known bug that I am not aware of?

My real code receive one input, process it and store it. It is the user who loops over the inputs. This sample shows me that detecting an allocation failure is not the right way to it since it is already too late. Also have you any ideas on how to prevent such a case? Checking the amount of free memory before each allocation would certainly kill the performances.

Also, isn't the OS supposed to keep a reserve of free memory to prevent a single program to consume it all?

This seems like a textbook case. However I was unable to find any similar code causing that error.

Upvotes: 2

Views: 245

Answers (1)

pah
pah

Reputation: 4778

The memory management behavior will differ from each operating system. I am not familiar with the memory management unit implemented in the Windows 7 operating system, but there's little you can do (from the kernel point of view) when resource starvation occur. Usually, the operating system will prefer to kill the process that is requesting a lot of resources, and this is probably what's happening in your case.

You've pointed out a "reserved region" behavior, but this will not solve the problem, even if it is implemented, because if the region is reserved, you still need to decide which processes will use it, and which processes will not, causing the processes that will not be allowed to use it to starve (and stop). Of course that the kernel should be cleaver enough to reserve memory to its own routines in order to prevent itself from crashing (so I don't know why windows kernel is crashing, but as I pointed out, I'm not familiar with its implementation).

I advise you to read some papers or books regarding memory management (specifically regarding windows memory management) from the point of view of the operating system. This will give you some leads for further research.

Hope this helps.

Upvotes: 1

Related Questions