Does .NET correctly handle memory paging with many small objects?

Question

I've several times developed applications with very large branched or linked structures of small objects (< 1kb each), either 1) simply creating the objects or 2) creating and accessing them.

In both instances, the application either stops completely or outright throws OutOfMemory once the physically available RAM us used up.

My understanding is that once physical RAM is exhausted, paging should occur, and, while very slow, the program should continue to work. In particular I am neither trying to allocate large objects, nor am I using arrays (or lists) of more than 2g objects (I'm not sure if that limit still applies nowadays).

I wrote a little test program which continuously allocated and stored 1GB blocks of memory. Since .NET seemed to delay the allocation, I also filled the blocks with data. I found that .NET correctly paged out to disk once RAM was used up, and the program became noticeably slower, but never crashed or OOM-ed.

So, why does .NET seem to have trouble with a lot of small objects? Is this particular to my configuration?

Answer 1

Windows correctly handles paging for .NET and any other process for that matter. Paging shouldn't really be a concern in the sense that it is Windows' job to decide which pages get to stay in physical memory versus the ones who gets paged to disk. Running out of physical memory is not a good sign though, since it means that all process in the system require frequent access to pages that don't really fit in physical memory.

OOM happens for other reasons:

• An x86 process running out of process address space on a 32 bit machine or on a 64 bit machine under WOW64. Basically such process cannot by default exceed 2GB. An x86 process might be able to use 3GB on a 32 bit machine using the /3GB option or under WOW64 if they have been flagged as large address aware.

  To find out if it is hitting the wall at 2GB, 3GB or 4GB, use the Virtual Size counter under Performance Monitor or download Process Explorer from Microsoft. I have tested this a few times and it always crash around 1,980,000KB of Virtual size. Please note that task manager only has Commit Size on Windows 7 and that is not as accurate as Virtual Size.

• Memory fragmentation could be a cause too, but I don't think it applies when you only have small objects that get compressed by the garbage collector.

• Running out of resources If your small objects use GDI handles, which are not managed, by default you could run out of them around 10000. In such case your will get the OOM error accompanied by the Red X (cross) on one or more part of the screen.

The Commit Size encompasses all the pages that have been allocated and are in use (regardless of being paged out or in RAM at this point). It represents the process current demand for memory. The Virtual Size is the full process address space which includes the committed and reserved segments. The latter are placeholder addresses that have been reserved for future use, but haven't been used or allocated yet (not even in the paging file).

Answer 2

I wrote a little program to confirm that, as Brian Rasmussen's pointed out, .NET is very well able to page out to disk.

Trivial program which keeps allocating chunks of RAM for testing system behavior.

Allocation will occur in chunks starting at 1GB size. The number of chunks is unlimited. In case an OutOfMemory exception is thrown, the size will be reduced. Each chunk will filled with data to ensure its memory pages were actually created and initialized.

In particular, running on a x64 box, you should be able to observe that .NET is able page out to disk nicely after filling up available physical RAM.

Please note that this will very likely slow down your system severly, possibly crashing other running applications in the process (web browsers are notorious for this).

Answer 3

Paging is a feature of the OS, so this is not something .NET does or doesn't do.

A common source of OOM is memory fragmentation. Remember, you don't actually allocate anything in a managed application. You just create objects. The runtime allocates memory necessary to store those objects. These allocations are done in chunks called segments. These are allocated as contiguous memory. Fragmentation can lead to situations where there's not enough contiguous memory to honor allocation of a new segment. If a segment cannot be allocated the runtime throws OOM.

The other common source of OOM is address space exhaustion. A common misunderstanding is that as long as there is enough memory available on the system, OOM should not occur. That's not the case for 32 bit apps. They have an address space of 2GB (or 4GB if on 64 bit and large memory address aware). Anything above that will trigger OOM regardless of how much memory the system might have. Since your question is tagged x64 I'm assuming that this is a 64 bit application in which case address space exhaustion is probably not the case.

As it stands your question does contain enough information to say why you see OOM exceptions. Are you storing all those small objects in a list or other structure? If so you might be using very large arrays as many of the collection classes are implemented using arrays.