Virtual Size causing program to run out of memory

Question

A server application I'm working on built using C++ on Windows runs out of memory when Virtual Size reaches somewhere around 2GB (32-bit application, with large address aware enabled). I notice, however, that Private Bytes is significantly smaller. The current statistics are:

Virtual Size: 2.6GB Private Bytes: 1.6GB

The difference in these two numbers is 1GB. So my questions are:

1. What does this 1GB difference represent?
2. Is my application running out of memory due to Virtual Size or Private Bytes?

I've also run my application through the VMMap utility and I notice that "Private Data" is usually an order of magnitude higher than the committed size. In other words, total size for Private Data might be 200MB, but the committed size is only 20MB. I'm not really sure what private data is, but based on my research so far it seems to indicate it's just part of the heap.

EDIT:

I've looked for memory leaks using Purify but I've not really found anything useful. Memory leaks in the form of memory without pointers doesn't seem to be the problem, but memory leaks in terms of memory being held onto for too long might be an issue, i haven't looked into that yet. However, the key is understanding why Virtual Size is causing the out of memory issues. Question #1 is the most important for me to understand this.

Answer 1

I had similar issue on my machine where the C/C++/.NET win32 application run out of memory. It consumed all 2GB of Virtual Addresses. It looked like Virtual Addresses Exhaustion because Process Explorer showed only about 900MB memory taken by the application. VMMap showed 1.6GB of Private Data and about 700MB uncommitted memory.

The reason was more than simple. The Application Verifier (C:\Windows\SysWOW64\appverif.exe) which was configured to test the application (Basics tests were marked - vfbasics.dll). After removing the application from Application Verifier it worked fine.

Answer 2

This is going to require a little explanation, so stick with me here.

First off, this topic is a confusing quagmire of conflicting terms, so please throw away all notions you have of "virtual memory" being anything to do with disks.

• Physical memory is memory stored on a physical device. This usually refers to system RAM, but can also be disk buffers, NIC buffers, graphics card VRAM, etc.
• Virtual memory is a set of physical address ranges mapped into user-mode (virtual) address ranges so that memory can be accessed in a safe and compartmentalised way.

A quick note on why we do this: if we gave processes direct memory addresses, we could only (feasibly) have a single memory store. This is inconvenient and bad for performance. When a virtual address translates to a physical address outside the range of the system memory (RAM), the processor issues a page fault. This signals an interrupt handler in the OS, which can then delegate the memory access operation to a different device. Useful!

On a 32-bit Windows system, the maximum amount of virtual memory that a process can address at any one point in time is 2GB. This can be increased to 3GB using AWE, or 4GB with /4GT and AWE. This does not mean that a process can only ever allocate 2GB (or 3GB / 4GB depending on the previously discussed settings) of memory. It just means that it cannot have concurrent access to more than that.

For example, if you open a memory-mapped file that is 1GB in size, your virtual memory usage will increase by 1GB. You're not touching the RAM, nor the disk, but you've allocated a block of virtual address space to your process. If you then want to allocate 1.1GB of RAM at the same time as having this memory mapped file available, you cannot. You must first unmap the file from your process. Remember that memory can still remain allocated and full of data, but not actually mapped into your process. If you have 8GB of RAM on your machine, you could fill 6GB of it up with data and map 2GB of it into your process. When you need to use a different section of that memory, you must unmap the existing block and map the other section.

So, onto the difference you're seeing:

1. Private bytes tells you how many bytes of virtual device memory your process has mapped, excluding virtual memory shared with other processes (e.g. mapped files, global heap, etc).

2. Working set tells you how many bytes of physical memory you are actively using. This includes physical memory, device buffers, and mapped files. It's a pretty strange figure, since it equates to touched physical memory + mapped virtual non-system memory. In general, you should completely ignore this figure. It's practically useless for debugging memory leaks.

3. Virtual bytes is the total amount of virtual memory you have mapped.

The difference is that you've mapped shared virtual memory, for example a bunch of DLL files or a block of global heap, into your process. The difference indicates that the total size of these shared mappings is roughly 1GB.

Keep in mind that none of this has anything to do with swapping, which is the transfer of system memory pages to disk (the so-called "pagefile") to increase the availability of fast system resources (RAM). The naming of that file has caused no end of confusion in this area of Windows, and I'll be elated when Microsoft finally decide to call it "swap" rather than "virtual memory" or "page file".

Answer 3

Just a comment: Enabling large address aware only signals the OS that it could safely divide the virtual address space of processes, created from that particular executable, in the 3:1 fashion instead of the usual 2:2. There are various reasons of why the application should explicitly identify that it supports 3:1 division but the most obvious one is that in 2:2 mode probing the MSB of a virtual address could be used to test if the address belongs to the kernel portion of the virtual address space or to the user portion. In 3:1 this test is no longer valid since the MSB is also 1 for 1/3rd of the user portion of the virtual address space. This requres that drivers which this application might call into should also be aware of possible 3:1 division and should use another method to test if a given virtual address belongs to userspace of not.

You still have to explicitly tell the kernel to enable support for 3:1 VA space division as Roman R. has pointed out.

Answer 4

1. Virtual Size vs Private Bytes explained: What is private bytes, virtual bytes, working set? (see quote below)
2. Your application is likely to hit Virtual Size limit at 2 GB, esp. since you are seeing such behavior yourself
3. /LARGEADDRESSAWARE extends the Virtual Size limit for your application in Win32 operating systems only when the system itself booted with /3GB AKA 4GT enabled

Virtual Bytes are the total virtual address space occupied by the entire process. This is like the working set, in the sense that it includes memory-mapped files (shared DLLs), but it also includes data in the standby list and data that has already been paged out and is sitting in a pagefile on disk somewhere. The total virtual bytes used by every process on a system under heavy load will add up to significantly more memory than the machine actually has.

So the relationships are:

• Private Bytes are what your app has actually allocated, but include pagefile usage;
• Working Set is the non-paged Private Bytes plus memory-mapped files;
• Virtual Bytes are the Working Set plus paged Private Bytes and standby list.