Memory management for intentionally memory intensive applications

Question

Note: I am aware of the question Memory management in memory intensive application, however that question appears to be about applications that make frequent memory allocations, whereas my question is about applications intentionally designed to consume as much physical memory as is safe.

I have a server application that uses large amounts of memory in order to perform caching and other optimisations (think SQL Server). The application runs on a dedicated machine, and so can (and should) consume as much memory as it wants / is able to in order to speed up and increase throughput and response times without worry of impacting other applications on the system.

The trouble is that if memory usage is underestimated, or if load increases its possible to end up with nasty failures as memory allocations fail - in this situation obviously the best thing to do is to free up memory in order to prevent the failure at the expense of performance.

Some assumptions:

• The application is running on a dedicated machine
• The memory requirements of the application exceed the physical memory on the machine (that is, if additional memory was available to the application it would always be able to use that memory to in some way improve response times or throughput)
• The memory is effectively managed in a way such that memory fragmentation is not an issue.
• The application knows what memory can be safely freed, and what memory should be freed first for the least performance impact.
• The app runs on a Windows machine

My question is - how should I handle memory allocations in such an application? In particular:

• How can I predict whether or not a memory allocation will fail?
• Should I leave a certain amount of memory free in order to ensure that core OS operations remain responsive (and don't in that way adversely impact the applications performance), and how can I find out how much memory that is?

The core objective is to prevent failures as a result of using too much memory, while at the same time using up as much memory as possible.

I'm a C# developer, however my hope is that the basic concepts for any such app are the same regardless of the language.

Answer 1

Your question reminds me of an old discussion "So what's wrong with 1975 programming ?". The architect of varnish-cache argues, that instead of telling the OS to get out of the way and manage all memory yourself, you should rather cooperate with the OS and let it understand what you intend to do with memory.

For example, instead of simply reading data from disk, you should use memory-mapped files. This allows the OS to apply its LRU algorithm to write-back data to disk when memory becomes scarce. At the same time, as long as there is enough memory, your data will stay in memory. Thus, your application may potentially use all memory, without risking getting killed.

Answer 2

That is a very good question, and bound to be subjective as well, because the very nature of the fundamental of C# is that all memory management is done by the runtime, i.e. Garbage Collector. The Garbage Collector is a non-deterministic entity that manages and sweeps the memory for reclaiming, depending on how often the memory gets fragmented, the GC will kick in hence to know in advance is not easy thing to do.

To properly manage the memory sounds tedious but common sense applies, such as the using clause to ensure an object gets disposed. You could put in a single handler to trap the OutOfMemory Exception but that is an awkward way, since if the program has run out of memory, does the program just seize up, and bomb out, or should it wait patiently for the GC to kick in, again determining that is tricky.

The load of the system can adversely affect the GC's job, almost to a point of a Denial of Service, where everything just grind to a halt, again, since the specifications of the machine, or what is the nature of that machine's job is unknown, I cannot answer it fully, but I'll assume it has loads of RAM..

In essence, while an excellent question, I think you should not worry about it and leave it to the .NET CLR to handle the memory allocation/fragmentation as it seems to do a pretty good job.

Hope this helps, Best regards, Tom.

Answer 3

In linux, the memory usage percentage is divided into following levels.

0 - 30% - no swapping 30 - 60% - swap dirty pages only 60 - 90% - swap clean pages also based on LRU policy.

90% - Invoke OOM(Out of memory) killer and kill the process consuming maximum memory.

check this - http://linux-mm.org/OOM_Killer

In think windows might have similar policy, so you can check the memory stats and make sure you never get to the max threshold.

One way to stop consuming more memory is to go to sleep and give more time for memory cleanup threads to run.