CODEWITHSUNDEEP
c#dockermemory
Nikita Sivukhin
Nikita Sivukhin

Reputation: 2605

Running .NET Core application in Docker container with memory limit

I testing different limits of Docker containers with the help of C#. For testing purposes, I write a simple program that generates different types of loads (CPU-bound work, memory allocations, network requests, etc).

For the scenarious with many memory allocations I write following code:

try {
    var chunks = new byte[10][];
    for (var chunkId = 0; chunkId < 10; chunkId++) {
        Console.WriteLine($"Chunk: {chunkId}");
        var chunk = new byte[100 * 1024 * 1024];
        for (var i = 0; i < chunk.Length; i++)
            chunk[i] = (byte) (i % 256);
        chunks[chunkId] = chunk;
    }
}
catch (Exception) { // To capture OutOfMemoryException and pause the process
    Thread.CurrentThread.Join();
}

This code allocated 10 chunks of 100MB contiguous segments of memory (1GB in total), but the magic happens when I run it inside the container with memory restrictions:

$> docker run --memory=1g --memory-swap=1g <workload-image>

In the container with RAM limited to 1GB, my application can allocate only 6-7 chunks of 100MB, and request to the new byte array failed with OutOfMemoryException.

At first, I thought, that CLR consumes memory for some internal structures, but after hours of digging into the memory dumps, I don't find any suspicious large objects in the .NET heap.

Finally, I set up the following simple experiment. I run my application in the Docker container with a 1GB RAM limit, and after OutOfMemoryException is handled in my catch statement and process paused, I run another copy of my program, which suddenly can allocate 3 additional chunks of memory (300MB).

I can't explain why my application can't allocate all chunks (or at least 9/10 chunks) with 1GB RAM limit. Can anybody shed the light on this mysterious behavior of .NET?

Problem reproduced with Docker Desktop (Windows 10, 5GB RAM of memory preallocated) and docker 19.03.11 in Ubuntu 18.10. My project targets netcoreapp3.1 and final image based on the mcr.microsoft.com/dotnet/core/sdk:3.1.300. I push my image in the Docker hub registry, if this can be helpful in the investigation of problem.

UPD: I write a simple analog of C# application in C++ and it successfully runs with 1GB limits. So, I think this is something tightly related to the .NET CLR.

Also, in the container application consumes 724 MB according to top (RES column):

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND    
  1 root      20   0 5749076 741512  19932 S   0.0   9.3   0:01.05 dotnet

Upvotes: 4

Views: 11479

Answers (1)

rob smith
rob smith

Reputation: 94

HeapHardLimit is 75% of container memory limit by default, see https://learn.microsoft.com/en-us/dotnet/core/run-time-config/garbage-collector#heap-limit

So with a 1GB container limit, you should be able to allocate 7x100MB chunks in a single dotnet process.

Upvotes: 7

Related Questions