How to find the maximum RAM that would be available to my process?

Question

I am trying to write a program to perform external merge sort on a massive dataset. As a first step, I need to split the dataset into chunks that would fit into RAM. I have the following questions:

1. Suppose my machine has x amount of RAM installed, is there a theoretical maximum limit on how much of it could be made available to my process?

   When I run the below program, I get a non-zero value as available memory when it fails. Why does the memory allocation fail when there is still unused RAM left? there is still 2.8GB free RAM when the memory allocation fails. What explains the observed behavior?

        List<string> list = new List<string>();
        try
        {
            while (true)
            {
                list.Add("random string");
            }
        }
        catch(Exception e)
        {
            Microsoft.VisualBasic.Devices.ComputerInfo CI = new ComputerInfo();
            Console.WriteLine(CI.AvailablePhysicalMemory);
        }

2. If there are other programs running concurrently, how do I determine, how much RAM is available for use by the current process?

Answer 1

Here is what you're looking for: ComputerInfo.AvailablePhysicalMemory

Gets the total amount of free physical memory for the computer.

private ulong GetMaxAvailableRAM()
{
    Microsoft.VisualBasic.Devices.ComputerInfo CI = new ComputerInfo();
    return CI.AvailablePhysicalMemory;            
}

NOTE: You will need a to add a reference to Microsoft.VisualBasic

UPDATE: Your sample to fill the RAM will run into a few other limits first.
It will first hit OutOfMemory if your not building in 64-bit. You should change your solution to build for x64 = 64-bit within the Solution configuration:

Secondly your List has a maximum supported array dimension. By adding many small objects you will hit that limit first. Here is a quick and dirty example making a List of Lists of strings.
(This could have smaller code using Images etc... But I was trying to stay similar to your example.)

When this is run it will consume all of your RAM and eventually start paging to disk. Remember Windows has Virtual RAM which will eventually get used up, but it's much slower than regular RAM. Also, if it uses all that up, then it might not even be able to allocate the space to instantiate the ComputerInfo Class.

NOTE: Be careful, this code will consume all RAM and potentially make your system unstable.

List<List<string>> list = new List<List<string>>();
try
{
    for (UInt32 I = 0; I < 134217727; I++)
    {
        List<string> SubList = new List<string>();

        list.Add(SubList);

        for (UInt32 x = 0; x < 134217727; x++)
        {
            SubList.Add("random string");
        }
    }
}
catch (Exception Ex)
{
    Console.WriteLine(Ex.Message);
    Microsoft.VisualBasic.Devices.ComputerInfo CI = new ComputerInfo();
    Console.WriteLine(CI.AvailablePhysicalMemory);
}

NOTE: To prevent using the Disk you could try to use something like System.Security.SecureString which prevents itself from being written to disk, but it would be very very slow to accumulate enough to fill your RAM.

Here is a test run showing the Physical Memory usage. I started running at (1)

I suggest for your final implementation that you use the ComputerInfo.AvailablePhysicalMemory value to determine how much of your data you can load before loading it (leaving some for the OS). And also look to lock objects in memory (usually used for Marshaling, etc..) to prevent accidental use of Virtual Memory.