C# ASP.NET MemoryCache grows excessively (23GB instead of 200MB) on one server – potential causes?

Question

I have a ASP.NET application (.NET8) that receives 100 REST calls per minute, each transferring 1MB of data, which is serialized (JSON) and stored in MemoryCache (Microsoft.Extensions.Caching.Memory). Most systems with identical hardware use around 300MB of memory. However, on one server (Windows Server 2019), the RAM usage increases continuously, eventually reaching 23GB (max memory of the system) and never being released.

I know that saving such big objects in RAM is not perfect, but i am not allowed to use an external caching service. I also use MessagePack or Newtonsoft to destroy the reference of my objects in some parts of my code.

What system/configuration factors could cause excessive memory growth on one server while others remain stable? Has anyone an idea what i can do to find this problem?

I suspect the issue is related to Large Object Heap (LOH) fragmentation and Garbage Collection (GC). Possible causes I’ve considered include:

• LOH fragmentation: Regular creation of 1MB objects leading to LOH issues.
• GC settings: Server GC vs. Workstation GC, or differences in configurations.
• Cache configuration: Old entries not being cleared from MemoryCache. I have limited the number of entries to 300. MemoryCache uses this limit, but the memory still increases
• System/environment differences: Variations in Windows settings (e.g., memory compression, pagefile, background processes).

By "to destroy references in my code": I mean

public static T? DestroyReference<T>(T data)   
{       
    return MessagePackSerializer.Deserialize<T?>(
           MessagePackSerializer.Serialize(data, MessagePackOptions),
           MessagePackOptions);   
}

Answer 1

It looks like you are trying to clone some very large object graphs, using either MessagePack or Newtonsoft, by serializing to huge, temporary byte arrays or strings. If the temporary arrays and strings exceed 85,000 bytes in length, you may be getting large object heap fragmentation. If so, you could try replacing the temporary memory with Microsoft.IO.RecyclableMemoryStream. This nuget from Microsoft is

A library to provide pooling for .NET MemoryStream objects to improve application performance, especially in the area of garbage collection.

To use it, install Microsoft.IO.RecyclableMemoryStream and instantiate a singleton RecyclableMemoryStreamManager for use in your app:

using Microsoft.IO;

public static partial class SerializationExtensions
{
    // You might want to inject this via dependency injection
    public static RecyclableMemoryStreamManager Manager { get; } = new RecyclableMemoryStreamManager( 
        new RecyclableMemoryStreamManager.Options
        {
            // Set any appropriate options here.
        });
}

Then, since you are using multiple serializers to create different cloning algorithms, define the following interface:

public interface ICloneWorker
{
    [return: NotNullIfNotNull(nameof(value))]
    T? Clone<T>(T? value);
}

And implementations using MessagePack and Newtonsoft:

public class MessagePackCloneWorker : ICloneWorker
{
    readonly MessagePackSerializerOptions? options = default;
    
    public MessagePackCloneWorker(MessagePackSerializerOptions? options = default) => this.options = options;

    [return: NotNullIfNotNull(nameof(value))]
    public T? Clone<T>(T? value)
    {
        if (value is null)
            return value;
        using Stream stream = SerializationExtensions.Manager.GetStream();
        MessagePackSerializer.Serialize(stream, value, options);
        stream.Position = 0;
        return MessagePackSerializer.Deserialize<T>(stream, options)!;
    }
}

public class NewtonsoftCloneWorker : ICloneWorker
{
    static JsonSerializerSettings DefaultNewtonsoftCloneSettings { get; } = new()
    {
        Formatting = Formatting.None,
        NullValueHandling = NullValueHandling.Ignore,
        DefaultValueHandling = DefaultValueHandling.Ignore,
        DateParseHandling = DateParseHandling.None,
    };
    readonly JsonSerializerSettings settings;
    
    public NewtonsoftCloneWorker(JsonSerializerSettings? settings = default) => this.settings = settings ?? DefaultNewtonsoftCloneSettings;

    [return: NotNullIfNotNull(nameof(value))]
    public T? Clone<T>(T? value)
    {
        if (value is null)
            return value;
        var serializer = JsonSerializer.CreateDefault(settings);

        using Stream stream = SerializationExtensions.Manager.GetStream();
        using (var textWriter = new StreamWriter(stream, leaveOpen : true))
        {
            serializer.Serialize(textWriter, value);
        }
        stream.Position = 0;
        using (var textReader = new StreamReader(stream))
        using (var jsonReader = new JsonTextReader(textReader))
        {
            return serializer.Deserialize<T>(jsonReader)!;
        }
    }
}

And now you will be able to rewrite DestroyReference<T>(T data) as follows:

static MessagePackCloneWorker worker = new(MessagePackOptions);
public static T? DestroyReference<T>(T data) => worker.Clone(data);

Demo fiddle here.

Notes:

1. You don't indicate why your models are so large. If you models consist mainly of strings, you might try cloning using Newtonsoft's JToken as an intermediate representation:

   public T? Clone<T>(T? value)
   {
       if (value is null)
           return value;
       var serializer = JsonSerializer.CreateDefault(settings);
       return JToken.FromObject(value, serializer).ToObject<T>(serializer)!;
   }
   

   If you do, the original string memory will be reused rather than cloned. Demo fiddle #2 here.

2. If your models require 1MB per instance and also require constant cloning, you should reconsider your design. For instance, if you could make the majority of their contents immutable you would eliminate the need for most of the cloning.