Malloc sometimes fails immediately after a free

Question

I want to test my code in low memory situations. I wrote this function with setrlimit to limit the available memory :

unsigned short int oom_enable = 0;
char* _oomfill = NULL;

uint32_t oom_setup(uint32_t ramlimit)
{
    struct rlimit limit;

    /* Limit available RAM to 64MB for all suites/testcase/process/fork */
    /* to test OOM without exhausting the system's resources */
    limit.rlim_cur = ramlimit;
    limit.rlim_max = limit.rlim_cur;
    if (setrlimit(RLIMIT_AS, &limit) != 0) {
        fprintf (stderr,_("setrlimit() failed with errno=%d %s\n"), errno,strerror(errno));
        /* Better to abort than to permit RAM bombing */
        abort();
    }
    if (getrlimit(RLIMIT_AS, &limit) != 0) {
        fprintf (stderr,_("getrlimit() failed with errno=%d %s\n"), errno,strerror(errno));
        /* Better to abort than to potentially permit RAM bombing */
        abort();
    }
    return limit.rlim_cur;
}

Then, I call it at the begining of my program :

    /* Limit available RAM to 64MB for all suites/testcase/process/fork */
    /* to test OOM without exhausting the system's resources */
    ramlimit=64*1024*1024;
    printf ("Limit memory usage to %uMB : ",ramlimit/1024/1024);
    ramlimit = oom_setup(ramlimit);
    printf ("%uB (%uMB)\n",ramlimit,ramlimit/1024/1024);

Then, I have 2 macros, one to eat most of the memory immediately before running the function under test :

 * After this call, there is AT MOST (KB*1024) bytes  available but probably
 * less. This has to be called immediately before the function under test.
 * Nothing else can be executed, printf, assertions, ....  */
#define OOMTEST_BEGIN(KB) \
    if (1==oom_enable) { \
        struct rlimit limit; \
        uint32_t min,cur,max; \
        if (NULL!=_oomfill) { \
            free(_oomfill); \
            _oomfill = NULL; \
        } \
        if (getrlimit(RLIMIT_AS, &limit) != 0) { \
            fprintf (stderr,_("getrlimit() failed with errno=%d %s\n"), errno,strerror(errno)); \
            abort(); \
        } \
        min = 0; \
        cur = 0; \
        max = limit.rlim_cur; \
        while ((max-min)>(KB*1024)) { \
            if (NULL!=_oomfill) \
                free(_oomfill); \
            cur = ((min+max)/2); \
            if (NULL==(_oomfill = malloc(cur))) { \
                max = cur; \
            } else { \
                min = cur; \
            } \
        } \
        fprintf(stderr,"OOM Test, consummed %u bytes.\n",cur); \
        /* Keep some minimal headroom (10B) for tooling */ \
        free(_oomfill); \
        if (NULL == (_oomfill=malloc(cur-10))) { \
            fprintf(stderr,_("OOM failed to keep headroom RAM.\n")); \
            abort(); \
        } \
        _oomfill[0]=1; \
    }

And the other to free the memory immediately after the function under test :

/** Free the RAM consumed by OOMTEST_BEGIN
 *
 * This has to be called IMMEDIATELY after the function under testing, BEFORE
 * any check, test, assertion, output of the results. */
#define OOMTEST_END \
    if (1==oom_enable) { \
        if (NULL!=_oomfill) { \
            free(_oomfill); \
            _oomfill = NULL; \
            malloc_trim(0); \
            /* SW barrier (compiler only) */ \
            __asm__ volatile("": : :"memory"); \
            /* HW barrier (CPU instruction) */ \
            __sync_synchronize(); \
        } \
    }

My test code looks like :

    size_t result;

    /* This function will trigger getblocksize when memtrack is still not initialized to test autoinit */
    OOMTEST_BEGIN(0.01);
    result=memtrack_getblocksize(NULL);
    OOMTEST_END;

    /* Check expected results */
    ck_assert_msg(0==result,"Blocksize of NULL pointer should be 0");

Everything works as expected, but sometimes The code stops at abort(), when _oomfill is cur bytes long, filling the whole RAM, if freeed, after the malloc(cur-10) attempt.

Basically, I try to malloc(fullRAM-10) after free(fullRAM). I works most of the time, but not in some situations. It is very difficult to debug in gdb/ddd and I would appreciate help or clues.

Edit: I tried to use realloc instead of free/malloc, same result. Edit: It was a bug in the logic : if malloc(cur) fails in the loop, max takes cur value and cur for the next iteration. But if max-min is below the required threshold, there is no "next iteration", _oomfill is NULL, and.... cur can be more than the available RAM, failing to malloc(cur-10). My fix, despite not perfect (could theorically loop forever) was to change the while condition :

while ((NULL==_oomfill)||((max-min)>(threshold)))

Thus, even if max-min fullfil the threshold constraint, the loop will continue to iterate until findind a valid value for cur.

Answer 1

One possible explanation is malloc may use different allocation strategies when you pass certain thresholds: it might be possible to allocate exactly 128KB (131072 bytes) because malloc uses memory mapping for sizes greater or equal to this threshold, and yet fail to allocate 131071 or even 131056 bytes because the overhead for arena based allocation may cause the system limit to be reached for this alternative path.

On systems using the GNU libc, this threshold can be customized using the M_MMAP_THRESHOLD environment variable.

You can further customize the GNU allocator via environment variables described here.

Also notice that your approach limits the maximum number of pages to map for the process, which interacts in subtle ways with the memory allocation routines: mapping threshold but also fragmentation may cause more memory pages to be required for some allocations than strictly necessary.

Depending on the allocation strategy, fragmentation can cause allocation failures if the required block length cannot be found in a sparse arena with small blocks distributed in an adverse fashion.

Answer 2

setrlimit(RLIMIT_AS, &limit) doesn't only set the limit of the returned memory from malloc, but limits all the address space used by the program, including malloc overhead. To understand why malloc fails in your case, you really have to dig into the source code of the implementation you are using.

Calling malloc(cur - 10); after calling foo = malloc(cur); free(foo); may fail for any of (but not limited to) the following reasons:

• malloc may, for any reason, ask for additional memory from the OS to increase its memory pool, and fail.

• malloc may want to give you more memory than you asked for.

• It may chose to use more overhead than previous calls to malloc

• To prevent fragmentation, blocks of size cur - 10 may be allocated from another memory pool than blocks of size cur, and there is no more free memory in that pool.