How to retrieve a list of Memory Cache keys in asp.net core?

Question

To be succinct. Is possible list all register keys from Memory Cache in the .Net Core Web Application?

I didn't find anything in IMemoryCache interface.

Answer 1

As of .NET 7, they have changed the internals of the object and there is the new CoherentState private class which is a private field inside the MemoryCache instance and within the CoherentState field (_coherentState) you can access the EntriesCollection collection that you guys have been referencing. So, in order to get the list of keys you can do the following:

var coherentState = typeof(MemoryCache).GetField("_coherentState", BindingFlags.NonPublic | BindingFlags.Instance);

        var coherentStateValue = coherentState.GetValue(_memoryCache);

        var entriesCollection = coherentStateValue.GetType().GetProperty("EntriesCollection", BindingFlags.NonPublic | BindingFlags.Instance);

        var entriesCollectionValue = entriesCollection.GetValue(coherentStateValue) as ICollection;

        var keys = new List<string>();

        if (entriesCollectionValue != null)
        {
            foreach (var item in entriesCollectionValue)
            {
                var methodInfo = item.GetType().GetProperty("Key");

                var val = methodInfo.GetValue(item);

                keys.Add(val.ToString());
            }
        }

I have tested this locally and it works!

UPDATE .NET 8

This code stopped working when I tested it against .NET 8. The reason being is that EntriesCollection has been renamed to StringEntriesCollection. There is another collection of NonStringEntriesCollection but I am not covering that because I am assuming we will all have string keys.

Considering all of the above the code for .NET 8 would look like this:

private List<string> GetAllKeys()
{
    var coherentState = typeof(MemoryCache).GetField("_coherentState", BindingFlags.NonPublic | BindingFlags.Instance);

    var coherentStateValue = coherentState.GetValue(_memoryCache);

    var stringEntriesCollection = coherentStateValue.GetType().GetProperty("StringEntriesCollection", BindingFlags.NonPublic | BindingFlags.Instance);

    var stringEntriesCollectionValue = stringEntriesCollection.GetValue(coherentStateValue) as ICollection;

    var keys = new List<string>();

    if (stringEntriesCollectionValue != null)
    {
        foreach (var item in stringEntriesCollectionValue)
        {
            var methodInfo = item.GetType().GetProperty("Key");

            var val = methodInfo.GetValue(item);

            keys.Add(val.ToString());
        }
    }

    return keys;
}

UPDATE .NET 9

On this version we have available the Keys property on the MemoryCache type that exposes an IEnumerable<object> Keys array which can simplify things. Nevertheless as far as I can tell, the code I have on version .NET 8 is still compatible. I am using the interface IMemoryCache on my MemoryCacheWrapper so I am not targeting the new property for now. Note that the Keys property is not available on the interface IMemoryCache

Answer 2

Update as of Feb 2024

Beginning with version 9.0.0-preview.1.24080.9 of the NuGet package Microsoft.Extensions.Caching.Memory, the type MemoryCache has a public property Keys.

The property was added with this commit.

---

Microsoft.Extensions.Caching.Memory.MemoryCache up to v9 of the nuget package does not expose any members allowing to retrieve all cache keys, although there is a way around the problem if we use reflection.

This answer is partially based upon the one by MarkM, adds some speed to the solution by reducing reflection usage to a minimum, adds support for Microsoft.Extensions.Caching.Memory versions 7 and 8, and packs everything into a single extension class.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Reflection;
using System.Reflection.Emit;

namespace Microsoft.Extensions.Caching.Memory;

public static class MemoryCacheExtensions
{
    #region Microsoft.Extensions.Caching.Memory_6_OR_OLDER

    private static readonly Lazy<Func<MemoryCache, object>> _getEntries6 =
        new(() => (Func<MemoryCache, object>)Delegate.CreateDelegate(
            typeof(Func<MemoryCache, object>),
            typeof(MemoryCache).GetProperty("EntriesCollection", BindingFlags.NonPublic | BindingFlags.Instance).GetGetMethod(true),
            throwOnBindFailure: true));

    #endregion

    #region Microsoft.Extensions.Caching.Memory_7_OR_NEWER

    private static readonly Lazy<Func<MemoryCache, object>> _getCoherentState =
        new(() => CreateGetter<MemoryCache, object>(typeof(MemoryCache)
            .GetField("_coherentState", BindingFlags.NonPublic | BindingFlags.Instance)));

    #endregion

    #region Microsoft.Extensions.Caching.Memory_7_TO_8.0.8

    private static readonly Lazy<Func<object, IDictionary>> _getEntries7 =
        new(() => CreateGetter<object, IDictionary>(typeof(MemoryCache)
            .GetNestedType("CoherentState", BindingFlags.NonPublic)
            .GetField("_entries", BindingFlags.NonPublic | BindingFlags.Instance)));

    #endregion

    #region Microsoft.Extensions.Caching.Memory_8.0.10_OR_NEWER

    private static readonly Lazy<Func<object, IDictionary>> _getStringEntries8010 =
        new(() => CreateGetter<object, IDictionary>(typeof(MemoryCache)
            .GetNestedType("CoherentState", BindingFlags.NonPublic)
            .GetField("_stringEntries", BindingFlags.NonPublic | BindingFlags.Instance)));

    private static readonly Lazy<Func<object, IDictionary>> _getNonStringEntries8010 =
        new(() => CreateGetter<object, IDictionary>(typeof(MemoryCache)
            .GetNestedType("CoherentState", BindingFlags.NonPublic)
            .GetField("_nonStringEntries", BindingFlags.NonPublic | BindingFlags.Instance)));

    #endregion

    private static Func<TParam, TReturn> CreateGetter<TParam, TReturn>(FieldInfo field)
    {
        var methodName = $"{field.ReflectedType.FullName}.get_{field.Name}";
        var method = new DynamicMethod(methodName, typeof(TReturn), [typeof(TParam)], typeof(TParam), true);
        var ilGen = method.GetILGenerator();
        ilGen.Emit(OpCodes.Ldarg_0);
        ilGen.Emit(OpCodes.Ldfld, field);
        ilGen.Emit(OpCodes.Ret);
        return (Func<TParam, TReturn>)method.CreateDelegate(typeof(Func<TParam, TReturn>));
    }

    private static readonly Func<MemoryCache, IEnumerable> _getKeys =
        FileVersionInfo.GetVersionInfo(Assembly.GetAssembly(typeof(MemoryCache)).Location) switch
        {
            { ProductMajorPart: < 7 } =>
                static cache => ((IDictionary)_getEntries6.Value(cache)).Keys,
            { ProductMajorPart: < 8 } or { ProductMajorPart: 8, ProductMinorPart: 0, ProductBuildPart: < 10 } =>
                static cache => _getEntries7.Value(_getCoherentState.Value(cache)).Keys,
            _ =>
                static cache => ((ICollection<string>)_getStringEntries8010.Value(_getCoherentState.Value(cache)).Keys)
                    .Concat((ICollection<object>)_getNonStringEntries8010.Value(_getCoherentState.Value(cache)).Keys)
        };

    public static IEnumerable GetKeys(this IMemoryCache memoryCache) =>
        _getKeys((MemoryCache)memoryCache);

    public static IEnumerable<T> GetKeys<T>(this IMemoryCache memoryCache) =>
        memoryCache.GetKeys().OfType<T>();
}

Usage:

var cache = new MemoryCache(new MemoryCacheOptions());
cache.GetOrCreate(1, ce => "one");
cache.GetOrCreate("two", ce => "two");

foreach (var key in cache.GetKeys())
    Console.WriteLine($"Key: '{key}', Key type: '{key.GetType()}'");

foreach (var key in cache.GetKeys<string>())
    Console.WriteLine($"Key: '{key}', Key type: '{key.GetType()}'");

// Output:

// Key: '1', Key type: 'System.Int32'
// Key: 'two', Key type: 'System.String'
// Key: 'two', Key type: 'System.String'

Notes:

• Reflection usage is reduced to a few calls that build the _getKeys delegate. When we're working with the retrieved MemoryCache keys, reflection is not used. In contrast to a raw reflection approach, this decreases execution time and saves machine resources during traversal of long collections of MemoryCache keys.
• In the solution we are casting MemoryCache's internal dictionary to IDictionary instead of the native ConcurrentDictionary<object, CacheEntry> because CacheEntry type is internal.
• The code was verified using a matrix of console and webapi apps based on .NET Framework 4.8, .NET 6, 7 and 8 with Microsoft.Extensions.Caching.Memory versions 6, 7, 8.0.0 and 8.0.1 (patched). LangVersion is 12.
• If you use this class in your code, consider covering it with tests like in this repository.

Answer 3

October 2024 Update. The below code is exactly as Roxton's solution. The only difference is I had to change _entries to _stringEntries and it worked.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Reflection.Emit;
using Microsoft.Extensions.Caching.Memory;

namespace Extensions
{
    public static class MemoryCacheExtensions
    {
        #region Microsoft.Extensions.Caching.Memory_6_OR_OLDER

        private static readonly Lazy<Func<MemoryCache, object>> GetEntries6 =
            new Lazy<Func<MemoryCache, object>>(() => (Func<MemoryCache, 
            object>)Delegate.CreateDelegate(
            typeof(Func<MemoryCache, object>),
            typeof(MemoryCache).GetProperty("EntriesCollection", BindingFlags.NonPublic | BindingFlags.Instance).GetGetMethod(true),
            throwOnBindFailure: true));

        #endregion

        #region Microsoft.Extensions.Caching.Memory_7_OR_NEWER

        private static readonly Lazy<Func<MemoryCache, object>> GetCoherentState =
            new Lazy<Func<MemoryCache, object>>(() =>
                CreateGetter<MemoryCache, object>(typeof(MemoryCache)
                .GetField("_coherentState", BindingFlags.NonPublic | BindingFlags.Instance)));

        private static readonly Lazy<Func<object, IDictionary>> GetEntries7 =
            new Lazy<Func<object, IDictionary>>(() =>
            CreateGetter<object, IDictionary>(typeof(MemoryCache)
                .GetNestedType("CoherentState", BindingFlags.NonPublic)
                .GetField("_stringEntries", BindingFlags.NonPublic | BindingFlags.Instance)));

        private static Func<TParam, TReturn> CreateGetter<TParam, TReturn>(FieldInfo field)
        {
            var methodName = $"{field.ReflectedType.FullName}.get_{field.Name}";
            var method = new DynamicMethod(methodName, typeof(TReturn), new[] { typeof(TParam) }, typeof(TParam), true);
            var ilGen = method.GetILGenerator();
            ilGen.Emit(OpCodes.Ldarg_0);
            ilGen.Emit(OpCodes.Ldfld, field);
            ilGen.Emit(OpCodes.Ret);
            return (Func<TParam, TReturn>)method.CreateDelegate(typeof(Func<TParam, TReturn>));
        }

        #endregion

        private static readonly Func<MemoryCache, IDictionary> GetEntries =
            Assembly.GetAssembly(typeof(MemoryCache)).GetName().Version.Major < 7
            ? (Func<MemoryCache, IDictionary>)(cache => cache != null ? (IDictionary)GetEntries6.Value(cache) : new Dictionary<MemoryCache, IDictionary>()) 
            : cache => cache != null ? GetEntries7.Value(GetCoherentState.Value(cache)) : new Dictionary<MemoryCache, IDictionary>();

        public static ICollection GetKeys(this IMemoryCache memoryCache) =>
        GetEntries((MemoryCache)memoryCache).Keys;

        public static IEnumerable<T> GetKeys<T>(this IMemoryCache memoryCache) =>
        memoryCache.GetKeys().OfType<T>();
    }
}

Answer 4

Seems there is change in .NET 8.0.10, I had to do following changes to get keys.

Note: This works when key is string, if keys are object we have to call _nonStringEntries instated of _stringEntries

Refer PR changes: PR

using Microsoft.Extensions.Caching.Memory;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Reflection.Emit;

    public static class MemoryCacheExtensions
    {
        #region Microsoft.Extensions.Caching.Memory_6_OR_OLDER

        private static readonly Lazy<Func<MemoryCache, object>> GetEntries6 =
            new Lazy<Func<MemoryCache, object>>(() => (Func<MemoryCache, object>)Delegate.CreateDelegate(
                typeof(Func<MemoryCache, object>),
                typeof(MemoryCache).GetProperty("EntriesCollection", BindingFlags.NonPublic | BindingFlags.Instance).GetGetMethod(true),
                throwOnBindFailure: true));

        #endregion

        #region Microsoft.Extensions.Caching.Memory_7_OR_NEWER

        private static readonly Lazy<Func<MemoryCache, object>> GetCoherentState =
            new Lazy<Func<MemoryCache, object>>(() =>
                CreateGetter<MemoryCache, object>(typeof(MemoryCache)
                    .GetField("_coherentState", BindingFlags.NonPublic | BindingFlags.Instance)));

        private static readonly Lazy<Func<object, IDictionary>> GetEntries7 =
            new Lazy<Func<object, IDictionary>>(() =>
            {
                // Try to get the "_entries" field first
                var entriesField = typeof(MemoryCache)
                    .GetNestedType("CoherentState", BindingFlags.NonPublic)?
                    .GetField("_entries", BindingFlags.NonPublic | BindingFlags.Instance);

                // If "_entries" is null, fallback to "_stringEntries"
                if (entriesField == null)
                {
                    entriesField = typeof(MemoryCache)
                        .GetNestedType("CoherentState", BindingFlags.NonPublic)?
                        .GetField("_stringEntries", BindingFlags.NonPublic | BindingFlags.Instance);
                }

                // If a valid field is found, create a getter for it
                if (entriesField != null)
                {
                    return CreateGetter<object, IDictionary>(entriesField);
                }

                // Handle cases where both fields are not found (throw exception or return null)
                throw new InvalidOperationException("Unable to find '_entries' or '_stringEntries' field in MemoryCache.");
            });

        private static Func<TParam, TReturn> CreateGetter<TParam, TReturn>(FieldInfo field)
        {
            var methodName = $"{field.ReflectedType.FullName}.get_{field.Name}";
            var method = new DynamicMethod(methodName, typeof(TReturn), new[] { typeof(TParam) }, typeof(TParam), true);
            var ilGen = method.GetILGenerator();
            ilGen.Emit(OpCodes.Ldarg_0);
            ilGen.Emit(OpCodes.Ldfld, field);
            ilGen.Emit(OpCodes.Ret);
            return (Func<TParam, TReturn>)method.CreateDelegate(typeof(Func<TParam, TReturn>));
        }

        #endregion

        private static readonly Func<MemoryCache, IDictionary> GetEntries =
            Assembly.GetAssembly(typeof(MemoryCache)).GetName().Version.Major < 7
                ? (cache => (IDictionary)GetEntries6.Value(cache))
                : cache => GetEntries7.Value(GetCoherentState.Value(cache));

        public static ICollection GetKeys(this IMemoryCache memoryCache) =>
            GetEntries((MemoryCache)memoryCache).Keys;

        public static IEnumerable<T> GetKeys<T>(this IMemoryCache memoryCache) =>
            memoryCache.GetKeys().OfType<T>();
    }

Answer 5

This feature is added in .NET 9 (including back-port into older TFMs, since this is an out-of-band NuGet package) via the MemoryCache.Keys property.

So:

IMemoryCache cache...
if (cache is MemoryCache mc) // we need the concrete type
{
    foreach (object key in mc.Keys)
    {
        // ...
        DoWhatever(key);
    }
}

Answer 6

An alternative to reflection and relying on internal implementation of MemoryCache approach, you can create a wrapper over IMemoryCache let's say CacheManager that will handle putting/getting things to cache and track keys records like this:

private readonly List<string> _keys = new List<string>();

private void OnCacheEntryAdded(string key)
{
        _keys.Add(key);
}
private void OnCacheEntryRemoved(string key)
{
        _keys.Remove(key);
}

public IEnumerable<string> GetKeys()
{
        foreach (var key in _keys.ToArray())
        {
            if (!IsSet(key))
            {
                _keys.Remove(key);
                continue;
            }

            yield return key;
         }
 }

Answer 7

MarkM's answer didn't quite work for me, it wouldn't cast the results to an ICollection, but I took the idea and came up with this that works quite well for me. Hopefully it helps someone else out there too:

// 2022-12-06
// Updated to work with both .Net7 and previous versions.  Code can handle either version as-is.  
// Remove code as needed for version you are not using if desired.

// Define the collection object for scoping.  It is created as a dynamic object since the collection
// method returns as an object array which cannot be used in a foreach loop to generate the list.
dynamic cacheEntriesCollection = null;

// This action creates an empty definitions container as defined by the class type.  
// Pull the _coherentState field for .Net version 7 or higher.  Pull the EntriesCollection 
// property for .Net version 6 or lower.    Both of these objects are defined as private, 
// so we need to use Reflection to gain access to the non-public entities.  
var cacheEntriesFieldCollectionDefinition = typeof(MemoryCache).GetField("_coherentState", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);
var cacheEntriesPropertyCollectionDefinition = typeof(MemoryCache).GetProperty("EntriesCollection", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);


// .Net 6 or lower.  
// In these versions of .Net, the EntriesCollection is a direct property of the MemoryCache type
// definition, so we can populate our cacheEntriesCollection with the definition from Relection
// and the values from our MemoryCache instance.
if (cacheEntriesPropertyCollectionDefinition != null)
{
    cacheEntriesCollection = cacheEntriesPropertyCollectionDefinition.GetValue(instanceIMemoryCache);
}

// .Net 7 or higher.
// Starting with .Net 7.0, the EntriesCollection object was moved to being a child object of
// the _coherentState field under the MemoryCache type.  Same process as before with an extra step.
// Populate the coherentState field variable with the definition from above using the data in
// our MemoryCache instance.  Then use Reflection to gain access to the private property EntriesCollection.
if (cacheEntriesFieldCollectionDefinition != null)
{
    var coherentStateValueCollection = cacheEntriesFieldCollectionDefinition.GetValue(instanceIMemoryCache);
    var entriesCollectionValueCollection = coherentStateValueCollection.GetType().GetProperty("EntriesCollection", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance);
    cacheEntriesCollection = entriesCollectionValueCollection.GetValue(coherentStateValueCollection);
}

// Define a new list we'll be adding the cache entries to
List<Microsoft.Extensions.Caching.Memory.ICacheEntry> cacheCollectionValues = new List<Microsoft.Extensions.Caching.Memory.ICacheEntry>();

foreach (var cacheItem in cacheEntriesCollection)
{
    // Get the "Value" from the key/value pair which contains the cache entry   
    Microsoft.Extensions.Caching.Memory.ICacheEntry cacheItemValue = cacheItem.GetType().GetProperty("Value").GetValue(cacheItem, null);

    // Add the cache entry to the list
    cacheCollectionValues.Add(cacheItemValue);
}

// You can now loop through the cacheCollectionValues list created above however you like.

Answer 8

For .NET6/7+ see @roxton's excellent answer.
In case you are looking for a drop-in replacement of @MarkM's solution for .NET7:

private static readonly FieldInfo? cacheEntriesStateDefinition = typeof(MemoryCache).GetField("_coherentState", BindingFlags.NonPublic | BindingFlags.Instance);
private static readonly PropertyInfo? cacheEntriesCollectionDefinition = cacheEntriesStateDefinition?.FieldType.GetProperty("EntriesCollection", BindingFlags.NonPublic | BindingFlags.Instance);
        
public static IEnumerable<ICacheEntry>? GetKeysAsICacheEntries(this IMemoryCache cache)
{
    if (cacheEntriesStateDefinition == null || cacheEntriesCollectionDefinition == null)
    {
        return null;
    }

    dynamic? cacheEntriesCollection = cacheEntriesCollectionDefinition.GetValue(cacheEntriesStateDefinition.GetValue(cache));

    if (cacheEntriesCollection == null)
    {
        return null;
    }

    List<ICacheEntry> cacheCollectionValues = new();
    foreach (dynamic cacheItem in cacheEntriesCollection)
    {
        ICacheEntry cacheItemValue = cacheItem.GetType().GetProperty("Value").GetValue(cacheItem, null);
        cacheCollectionValues.Add(cacheItemValue);
    }

    return cacheCollectionValues;
}

Answer 9

I was using MemoryCacheExtensions, suggested in roxton’s answer to show keys list for lazyCache, that wraps MemoryCache as MemoryCacheProvider

public static IEnumerable<string> GetKeys(this IAppCache appCache)  
{
    var cacheProvider = appCache.CacheProvider as MemoryCacheProvider;
    if (cacheProvider != null) //may be MockCacheProvider in tests 
    {
    var field = typeof(MemoryCacheProvider).GetField("cache", BindingFlags.NonPublic | BindingFlags.Instance);
    var memoryCache = field.GetValue(cacheProvider) as MemoryCache;
    return memoryCache.GetKeys<string>();
    }
    return new List<string>();
}

Answer 10

I know it's not the answer, but... There is another approach to this, you could cache a list. Something like this:

public async Task<List<User>> GetUsers()
{
    var cacheKey = "getAllUserCacheKey";

    if (_usersCache != null && ((MemoryCache)_usersCache).Count > 0)
    {
        return _usersCache.Get<List<User>>(cacheKey);
    }

    var users = await _userRepository.GetAll();


    // Set cache options.
    var cacheEntryOptions = new MemoryCacheEntryOptions()
        .SetSlidingExpiration(TimeSpan.FromMinutes(5));

    _usersCache.Set(cacheKey, users);

    return users;

}

Answer 11

Currently there is no such method in the IMemoryCache interface to return all the cache keys. As per this github issue comments, i do not think that would be added in the future.

Quoting Eilons comment

I think it's doubtful this would be available because part of the idea with caching is that mere moments after you ask it a question, the answer could have changed. That is, suppose you have the answer to which keys are there - a moment later the cache is purged and the list of keys you have is invalid.

If you need the keys, you should maintain the list of keys in your app while you set items to the cache and use that as needed.

Here is another useful github issue

Will there be GetEnumerator() for MemoryCache ?

Answer 12

We implemented this concept to enable removing by regex pattern.

The full implementation is may be found in Saturn72 github repository. We are shifting to AspNet Core these days so the location may moved. Search for MemoryCacheManager in the repository This is the current location

Answer 13

There is no such thing in .Net Core yet. Here is my workaround:

 var field = typeof(MemoryCache).GetProperty("EntriesCollection", BindingFlags.NonPublic | BindingFlags.Instance);
 var collection = field.GetValue(_memoryCache) as ICollection;
 var items = new List<string>();
 if (collection != null)
 foreach (var item in collection)
 {
      var methodInfo = item.GetType().GetProperty("Key");
      var val = methodInfo.GetValue(item);
      items.Add(val.ToString());
 }