Thread safe caching mechanism (not .NET built in cache) ASPX C#

Question

I need to create an web module, with this module i need to fetch the title of some web site, after i will find that title i need to store that in thread safe caching mechanism and i need to save there the 10 lat fetched titles.

Any help please ?

Answer 1

Posted this same answer over at: Thread-safe cache libraries for .NET

I know your pain as I am one of the Architects of Dedoose. I have messed around with a lot of caching libraries and ended up building this one after much tribulation. The one assumption for this Cache Manager is that all collections stored by this class implement an interface to get a Guid as a "Id" property on each object. Being that this is for a RIA it includes a lot of methods for adding /updating /removing items from these collections.

Here's my CollectionCacheManager

public class CollectionCacheManager
{
    private static readonly object _objLockPeek = new object();
    private static readonly Dictionary<String, object> _htLocksByKey = new Dictionary<string, object>();
    private static readonly Dictionary<String, CollectionCacheEntry> _htCollectionCache = new Dictionary<string, CollectionCacheEntry>();

    private static DateTime _dtLastPurgeCheck;

    public static List<T> FetchAndCache<T>(string sKey, Func<List<T>> fGetCollectionDelegate) where T : IUniqueIdActiveRecord
    {
        List<T> colItems = new List<T>();

        lock (GetKeyLock(sKey))
        {
            if (_htCollectionCache.Keys.Contains(sKey) == true)
            {
                CollectionCacheEntry objCacheEntry = _htCollectionCache[sKey];
                colItems = (List<T>) objCacheEntry.Collection;
                objCacheEntry.LastAccess = DateTime.Now;
            }
            else
            {
                colItems = fGetCollectionDelegate();
                SaveCollection<T>(sKey, colItems);
            }
        }

        List<T> objReturnCollection = CloneCollection<T>(colItems);
        return objReturnCollection;
    }

    public static List<Guid> FetchAndCache(string sKey, Func<List<Guid>> fGetCollectionDelegate)
    {
        List<Guid> colIds = new List<Guid>();

        lock (GetKeyLock(sKey))
        {
            if (_htCollectionCache.Keys.Contains(sKey) == true)
            {
                CollectionCacheEntry objCacheEntry = _htCollectionCache[sKey];
                colIds = (List<Guid>)objCacheEntry.Collection;
                objCacheEntry.LastAccess = DateTime.Now;
            }
            else
            {
                colIds = fGetCollectionDelegate();
                SaveCollection(sKey, colIds);
            }
        }

        List<Guid> colReturnIds = CloneCollection(colIds);
        return colReturnIds;
    }


    private static List<T> GetCollection<T>(string sKey) where T : IUniqueIdActiveRecord
    {
        List<T> objReturnCollection = null;

        if (_htCollectionCache.Keys.Contains(sKey) == true)
        {
            CollectionCacheEntry objCacheEntry = null;

            lock (GetKeyLock(sKey))
            {
                objCacheEntry = _htCollectionCache[sKey];
                objCacheEntry.LastAccess = DateTime.Now;
            }

            if (objCacheEntry.Collection != null && objCacheEntry.Collection is List<T>)
            {
                objReturnCollection = CloneCollection<T>((List<T>)objCacheEntry.Collection);
            }
        }

        return objReturnCollection;
    }


    public static void SaveCollection<T>(string sKey, List<T> colItems) where T : IUniqueIdActiveRecord
    {

        CollectionCacheEntry objCacheEntry = new CollectionCacheEntry();

        objCacheEntry.Key = sKey;
        objCacheEntry.CacheEntry = DateTime.Now;
        objCacheEntry.LastAccess = DateTime.Now;
        objCacheEntry.LastUpdate = DateTime.Now;
        objCacheEntry.Collection = CloneCollection(colItems);

        lock (GetKeyLock(sKey))
        {
            _htCollectionCache[sKey] = objCacheEntry;
        }
    }

    public static void SaveCollection(string sKey, List<Guid> colIDs)
    {

        CollectionCacheEntry objCacheEntry = new CollectionCacheEntry();

        objCacheEntry.Key = sKey;
        objCacheEntry.CacheEntry = DateTime.Now;
        objCacheEntry.LastAccess = DateTime.Now;
        objCacheEntry.LastUpdate = DateTime.Now;
        objCacheEntry.Collection = CloneCollection(colIDs);

        lock (GetKeyLock(sKey))
        {
            _htCollectionCache[sKey] = objCacheEntry;
        }
    }

    public static void UpdateCollection<T>(string sKey, List<T> colItems) where T : IUniqueIdActiveRecord
    {
        lock (GetKeyLock(sKey))
        {
            if (_htCollectionCache.ContainsKey(sKey) == true)
            {
                CollectionCacheEntry objCacheEntry = _htCollectionCache[sKey];
                objCacheEntry.LastAccess = DateTime.Now;
                objCacheEntry.LastUpdate = DateTime.Now;
                objCacheEntry.Collection = new List<T>();

                //Clone the collection before insertion to ensure it can't be touched
                foreach (T objItem in colItems)
                {
                    objCacheEntry.Collection.Add(objItem);
                }

                _htCollectionCache[sKey] = objCacheEntry;
            }
            else
            {
                SaveCollection<T>(sKey, colItems);
            }
        }
    }

    public static void UpdateItem<T>(string sKey, T objItem)  where T : IUniqueIdActiveRecord
    {
        lock (GetKeyLock(sKey))
        {
            if (_htCollectionCache.ContainsKey(sKey) == true)
            {
                CollectionCacheEntry objCacheEntry = _htCollectionCache[sKey];
                List<T> colItems = (List<T>)objCacheEntry.Collection;

                colItems.RemoveAll(o => o.Id == objItem.Id);
                colItems.Add(objItem);

                objCacheEntry.Collection = colItems;

                objCacheEntry.LastAccess = DateTime.Now;
                objCacheEntry.LastUpdate = DateTime.Now;
            }
        }
    }

    public static void UpdateItems<T>(string sKey, List<T> colItemsToUpdate) where T : IUniqueIdActiveRecord
    {
        lock (GetKeyLock(sKey))
        {
            if (_htCollectionCache.ContainsKey(sKey) == true)
            {
                CollectionCacheEntry objCacheEntry = _htCollectionCache[sKey];
                List<T> colCachedItems = (List<T>)objCacheEntry.Collection;

                foreach (T objItem in colCachedItems)
                {
                    colCachedItems.RemoveAll(o => o.Id == objItem.Id);
                    colCachedItems.Add(objItem);
                }

                objCacheEntry.Collection = colCachedItems;

                objCacheEntry.LastAccess = DateTime.Now;
                objCacheEntry.LastUpdate = DateTime.Now;
            }
        }
    }

    public static void RemoveItemFromCollection<T>(string sKey, T objItem) where T : IUniqueIdActiveRecord
    {
        lock (GetKeyLock(sKey))
        {
            List<T> objCollection = GetCollection<T>(sKey);
            if (objCollection != null && objCollection.Count(o => o.Id == objItem.Id) > 0)
            {
                objCollection.RemoveAll(o => o.Id == objItem.Id);
                UpdateCollection<T>(sKey, objCollection);
            }
        }
    }

    public static void RemoveItemsFromCollection<T>(string sKey, List<T> colItemsToAdd) where T : IUniqueIdActiveRecord
    {
        lock (GetKeyLock(sKey))
        {
            Boolean bCollectionChanged = false;

            List<T> objCollection = GetCollection<T>(sKey);
            foreach (T objItem in colItemsToAdd)
            {
                if (objCollection != null && objCollection.Count(o => o.Id == objItem.Id) > 0)
                {
                    objCollection.RemoveAll(o => o.Id == objItem.Id);
                    bCollectionChanged = true;
                }
            }
            if (bCollectionChanged == true)
            {
                UpdateCollection<T>(sKey, objCollection);
            }
        }
    }

    public static void AddItemToCollection<T>(string sKey, T objItem) where T : IUniqueIdActiveRecord
    {
        lock (GetKeyLock(sKey))
        {
            List<T> objCollection = GetCollection<T>(sKey);
            if (objCollection != null && objCollection.Count(o => o.Id == objItem.Id) == 0)
            {
                objCollection.Add(objItem);
                UpdateCollection<T>(sKey, objCollection);
            }
        }
    }

    public static void AddItemsToCollection<T>(string sKey, List<T> colItemsToAdd) where T : IUniqueIdActiveRecord
    {
        lock (GetKeyLock(sKey))
        {
            List<T> objCollection = GetCollection<T>(sKey);
            Boolean bCollectionChanged = false;
            foreach (T objItem in colItemsToAdd)
            {
                if (objCollection != null && objCollection.Count(o => o.Id == objItem.Id) == 0)
                {
                    objCollection.Add(objItem);
                    bCollectionChanged = true;
                }
            }
            if (bCollectionChanged == true)
            {
                UpdateCollection<T>(sKey, objCollection);
            }
        }
    }

    public static void PurgeCollectionByMaxLastAccessInMinutes(int iMinutesSinceLastAccess)
    {
        DateTime dtThreshHold = DateTime.Now.AddMinutes(iMinutesSinceLastAccess * -1);

        if (_dtLastPurgeCheck == null || dtThreshHold > _dtLastPurgeCheck)
        {

            lock (_objLockPeek)
            {
                CollectionCacheEntry objCacheEntry;
                List<String> colKeysToRemove = new List<string>();

                foreach (string sCollectionKey in _htCollectionCache.Keys)
                {
                    objCacheEntry = _htCollectionCache[sCollectionKey];
                    if (objCacheEntry.LastAccess < dtThreshHold)
                    {
                        colKeysToRemove.Add(sCollectionKey);
                    }
                }

                foreach (String sKeyToRemove in colKeysToRemove)
                {
                    _htCollectionCache.Remove(sKeyToRemove);
                }
            }

            _dtLastPurgeCheck = DateTime.Now;
        }
    }

    public static void ClearCollection(String sKey)
    {
        lock (GetKeyLock(sKey))
        {
            lock (_objLockPeek)
            {
                if (_htCollectionCache.ContainsKey(sKey) == true)
                {
                    _htCollectionCache.Remove(sKey);
                }
            }
        }
    }


    #region Helper Methods
    private static object GetKeyLock(String sKey)
    {
        //Ensure even if hell freezes over this lock exists
        if (_htLocksByKey.Keys.Contains(sKey) == false)
        {
            lock (_objLockPeek)
            {
                if (_htLocksByKey.Keys.Contains(sKey) == false)
                {
                    _htLocksByKey[sKey] = new object();
                }
            }
        }

        return _htLocksByKey[sKey];
    }

    private static List<T> CloneCollection<T>(List<T> colItems) where T : IUniqueIdActiveRecord
    {
        List<T> objReturnCollection = new List<T>();
        //Clone the list - NEVER return the internal cache list
        if (colItems != null && colItems.Count > 0)
        {
            List<T> colCachedItems = (List<T>)colItems;
            foreach (T objItem in colCachedItems)
            {
                objReturnCollection.Add(objItem);
            }
        }
        return objReturnCollection;
    }

    private static List<Guid> CloneCollection(List<Guid> colIds)
    {
        List<Guid> colReturnIds = new List<Guid>();
        //Clone the list - NEVER return the internal cache list
        if (colIds != null && colIds.Count > 0)
        {
            List<Guid> colCachedItems = (List<Guid>)colIds;
            foreach (Guid gId in colCachedItems)
            {
                colReturnIds.Add(gId);
            }
        }
        return colReturnIds;
    } 
    #endregion

    #region Admin Functions
    public static List<CollectionCacheEntry> GetAllCacheEntries()
    {
        return _htCollectionCache.Values.ToList();
    }

    public static void ClearEntireCache()
    {
        _htCollectionCache.Clear();
    }
    #endregion

}

public sealed class CollectionCacheEntry
{
    public String Key;
    public DateTime CacheEntry;
    public DateTime LastUpdate;
    public DateTime LastAccess;
    public IList Collection;
}

Here is an example of how I use it:

public static class ResourceCacheController
{
    #region Cached Methods
    public static List<Resource> GetResourcesByProject(Guid gProjectId)
    {
        String sKey = GetCacheKeyProjectResources(gProjectId);
        List<Resource> colItems = CollectionCacheManager.FetchAndCache<Resource>(sKey, delegate() { return ResourceAccess.GetResourcesByProject(gProjectId); });
        return colItems;
    } 

    #endregion

    #region Cache Dependant Methods
    public static int GetResourceCountByProject(Guid gProjectId)
    {
        return GetResourcesByProject(gProjectId).Count;
    }

    public static List<Resource> GetResourcesByIds(Guid gProjectId, List<Guid> colResourceIds)
    {
        if (colResourceIds == null || colResourceIds.Count == 0)
        {
            return null;
        }
        return GetResourcesByProject(gProjectId).FindAll(objRes => colResourceIds.Any(gId => objRes.Id == gId)).ToList();
    }

    public static Resource GetResourceById(Guid gProjectId, Guid gResourceId)
    {
        return GetResourcesByProject(gProjectId).SingleOrDefault(o => o.Id == gResourceId);
    }
    #endregion

    #region Cache Keys and Clear
    public static void ClearCacheProjectResources(Guid gProjectId)
    {            CollectionCacheManager.ClearCollection(GetCacheKeyProjectResources(gProjectId));
    }

    public static string GetCacheKeyProjectResources(Guid gProjectId)
    {
        return string.Concat("ResourceCacheController.ProjectResources.", gProjectId.ToString());
    } 
    #endregion

    internal static void ProcessDeleteResource(Guid gProjectId, Guid gResourceId)
    {
        Resource objRes = GetResourceById(gProjectId, gResourceId);
        if (objRes != null)
        {                CollectionCacheManager.RemoveItemFromCollection(GetCacheKeyProjectResources(gProjectId), objRes);
        }
    }

    internal static void ProcessUpdateResource(Resource objResource)
    {
        CollectionCacheManager.UpdateItem(GetCacheKeyProjectResources(objResource.Id), objResource);
    }

    internal static void ProcessAddResource(Guid gProjectId, Resource objResource)
    {
        CollectionCacheManager.AddItemToCollection(GetCacheKeyProjectResources(gProjectId), objResource);
    }
}

Here's the Interface in question:

public interface IUniqueIdActiveRecord
{
    Guid Id { get; set; }

}

Hope this helps, I've been through hell and back a few times to finally arrive at this as the solution, and for us It's been a godsend, but I cannot guarantee that it's perfect, only that we haven't found an issue yet.

Answer 2

Writing some locking code would be fairly easy except for...

How do you want to retrieve it? Do you want to be able to enumerate (foreach) over the list in a thread-safe fashion? There are a number of different ways to do that part, each with trade-offs.

You could go with the default behavior This probably won't work well -- you'll get an exception if someone changes the list while you are enumerating it.

You could lock the collection during the whole course of the enumeration. This means that any thread attempting to add to your cache will be blocked until the foreach loop exits.

You could copy the collection internally each time you enumerate it and enumerate the copy. This means that if someone adds to your list while you are enumerating it, you won't "see" the change.

For a list of ten, I'd go with the last option. (copy internally).

You code would look something like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Enumerator
{
    class Program
    {
        static void Main(string[] args)
        {
            MyCache<string> cache = new MyCache<string>();
            cache.Add("test");
            foreach (string item in cache)
                Console.WriteLine(item);
            Console.ReadLine();
        }
    }

    public class MyCache<T>: System.Collections.IEnumerable
    {
        private  readonly LinkedList<T> InternalCache = new LinkedList<T>();
        private  readonly object _Lock = new Object();

        public  void Add(T item)
        {
            lock (_Lock)
            {
                if (InternalCache.Count == 10)
                    InternalCache.RemoveLast();
                InternalCache.AddFirst(item);
            }
        }

        #region IEnumerable Members

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            // copy the internal cache to an array.  We'll really be enumerating that array
            // our enumeration won't block subsequent calls to Add, but this "foreach" instance won't see Adds either
            lock (_Lock)
            {
                T[] enumeration = new T[InternalCache.Count];
                InternalCache.CopyTo(enumeration, 0);
                return enumeration.GetEnumerator();
            }
        }

        #endregion

    }

}

<B>EDIT 1:</B> After sharing some comments with Rob Levine (below), I thought I'd throw a couple other alternatives out there.

This version allows you to iterate the collection lock-free. However, the Add() method is a little more expensive, as it must copy the list (moved the expense off of the Enumerate, and onto the add).

    public class Cache2<T>: IEnumerable<T>
    {
        // changes occur to this list, and it is copied to ModifyableList
        private LinkedList<T> ModifyableList = new LinkedList<T>();

        // This list is the one that is iterated by GetEnumerator
        private volatile LinkedList<T> EnumeratedList = new LinkedList<T>();

        private readonly object LockObj = new object();

        public void Add(T item)
        {
            // on an add, we swap out the list that is being enumerated
            lock (LockObj)
            {
                if (this.ModifyableList.Count == 10)
                    this.ModifyableList.RemoveLast();

                this.ModifyableList.AddFirst(item);
                this.EnumeratedList = this.ModifyableList;
                // the copy needs to happen within the lock, so that threaded calls to Add() remain consistent
                this.ModifyableList = new LinkedList<T>(this.ModifyableList);
            }

        }

        #region IEnumerable<T> Members

        IEnumerator<T> IEnumerable<T>.GetEnumerator()
        {
            IEnumerable<T> enumerable = this.EnumeratedList;
            return enumerable.GetEnumerator();
        }

        #endregion

        #region IEnumerable Members

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            System.Collections.IEnumerable enumerable = this.EnumeratedList;
            return enumerable.GetEnumerator();
        }

        #endregion
    }

<B>Edit 2:</B> In the last example, we had a really inexpensive iteration, with the trade-off being a more expensive call to Add(). Next, I thought about using a ReaderWriterLockSlim (this is a .Net 3.5 object -- the old ReaderWriterLock offered pretty poor performance)

With this model, the Add() method is less expensive than the previous model (although Add still has to take an exclusive lock). With this model, we don't have to create copies of the list. When we enumerate the list, we enter a readlock, which does not block other readers, but does block/is blocked by writers (calls to Add). As to which model is better -- it probably depends upon how you are using the cache. I would recommend Testing and measuring.

    public class Cache3<T> : IEnumerable<T>
    {
        private LinkedList<T> InternalCache = new LinkedList<T>();
        private readonly System.Threading.ReaderWriterLockSlim LockObj = new System.Threading.ReaderWriterLockSlim();

        public void Add(T item)
        {
            this.LockObj.EnterWriteLock();
            try
            {
                if(this.InternalCache.Count == 10)
                    this.InternalCache.RemoveLast();

                this.InternalCache.AddFirst(item);
            }
            finally
            {
                this.LockObj.ExitWriteLock();
            }
        }

        #region IEnumerable<T> Members

        IEnumerator<T> IEnumerable<T>.GetEnumerator()
        {
            this.LockObj.EnterReadLock();
            try
            {
                foreach(T item in this.InternalCache)
                    yield return item;
            }
            finally
            {
                this.LockObj.ExitReadLock();
            }
        }

        #endregion

        #region IEnumerable Members

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            this.LockObj.EnterReadLock();
            try
            {
                foreach (T item in this.InternalCache)
                    yield return item;
            }
            finally
            {
                this.LockObj.ExitReadLock();
            }
        }

        #endregion
    }

Answer 3

you might want to read up on this technique. Read-copy-update (RCU).