Can I 'reserve' memory somehow, so as to ensure malloc doesn't fail for this reason

Question

Is it possible to 'reserve' memory before a malloc() call? In other words, can I do something (perhaps OS-specific) which ensures there is a certain amount of free memory available, so that you know that your next malloc() (or realloc() etc.) call won't return NULL due to lack of memory?

The 'reservation' or 'pre-allocation' can fail just like a malloc, but if it succeeds, I want to be sure my next malloc() succeeds.

Notes:

• Yes, I know, I want to allocate memory before allocating memory. That's exactly right. The thing is the later allocations are not really under my control and I want to be able to assume they succeed.
• Bonus points for an answer regarding multi-threaded code as well.
• My motivation: I was considering adopting the use of glib for my C development, but apparently it abort()s when it fails to allocate memory, and that's not acceptable to me.
• Perhaps a solution which dynamically replaces the malloc symbol with something else? Or the symbol for the function wrapping the sbrk system call?

Answer 1

I realize the following does not directly answer your question with respect to malloc(). It is instead an attempt to offer up another avenue that might be applicable to your situation.

For a few years I was dealing with certified embedded systems. Two of the constraints were that 1) we were forbidden to free memory and 2) we were forbidden from allocating memory beyond a certain point during the initialization process. This was because fragmentation that could result from dynamic memory allocations and deallocations made it too costly to certify (and guarantee that allocations would succeed).

Our solution was to allocate pools of memory during the early initialization process. The blocks of memory handled by a given pool would all be the same size, thereby avoiding the fragmentation issue. Different pools would handle differently sized memory blocks for a different purpose. This meant that we had to allocate enough memory up front for our worst case memory consumption scenario as well as manage those pools ourselves.

Hope this helps.

Answer 2

With glibc you can hook the allocation functions:

https://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html

Now that you control memory allocation in your program, you can do what you like, including writing a function to reserve a (possibly thread-local, since you asked about multi-threading) chunk of memory from the system that future calls to your malloc and realloc hooks will use to return memory.

Obviously, you need to somehow know in advance an upper bound how much memory will be required by the series of malloc calls that you need to not fail.

Answer 3

Back in the old Mac Toolbox days it was extremely common to use a chunk of memory called a "rainy-day fund." You'd allocate enough memory such that if you freed it, there'd be enough free memory to throw up a dialog box explaining that the app had run out of memory, save your work, and exit. Then you'd keep that pointer around until malloc() returned null, and at least you'd be guaranteed to be able to deal with it gracefully.

That was on a 100% real-memory system, though, and things these days are very different. Still, if we're talking about those small and simple real-memory systems that still exist, then a similar strategy still makes sense.

Answer 4

Obviously there's no magic way for your program to ensure your system has an arbitrary amount of memory, but you can get the memory as soon as your process starts, so that it won't fail unexpectedly part way through the work/day when it'll be a right pain.

On some OSes, simply doing a big malloc then freeing the memory immediately will still have called sbrk or similar OS function to grow your process memory, but even that's not a great solution because getting virtual address space is still a ways short of getting physical memory to back it when needed, so you'd want to write through that memory with some noise values, then even if it's swapped out to disk while unused you can expect that the virtual memory of the system is committed to your memory needs and will instead deny other programs (or smaller new/malloc requests you make later ;-P) memory should the system be running short.

Another approach is to seek an OS specific function to insist on locking memory pages in physical memory, such as mlock(2) on Linux.

(These kind of "I'm the most important thing on the server" assumptions tend to make for a fragile system once a few running programs have all taken that attitude....)