How to compare IEnumerable<Dictionary<string, object>> based on matching keys in C#

Question

I want to compare data from two different lists, a source list & a target list. They are two different IEnumerable<Dictionary<string, object>>

I am running two tests and the result for each test should output the missing records from the target and assign the missing records to a new IEnumerable<Dictionary<string, object>>.

The keys in both Dictionary<string, object> are always the same. However, the length of the list and the values in the dictionary might not be.

List 1:

List 2:

Here is how I tried but it only compares the first key and value of the lists. I want to know if there are any different values with the same keys in each list.

My Controller:

    [HttpPost]
    public ActionResult TestResult(ValidationTestVM model)
    {

           model.NotFoundInSource = ServiceLayer.RecordCountResults(sourceQueryResults, targetQueryResults).ToList();

           model.NotFoundInTarget = ServiceLayer.RecordCountResults(targetQueryResults, sourceQueryResults).ToList();
     }

service class:

     public static IEnumerable<Dictionary<string, object>> RecordCountResults(IEnumerable<Dictionary<string, object>> source, IEnumerable<Dictionary<string, object>> target)
     {
            var results = new List<Dictionary<string, object>>();

            if((source != null && target != null) || (source.Count() > 0 && target.Count() > 0))
                results = source.Where(m =>!target.Select(s => s.First().Value).Contains(m.First().Value)).ToList();            

            return results;
     }

This does not seem to work correctly. Can someone tell what am I doing wrong?

A dummy method that creates the sample data and showing how I expect to see the returns lists:

        public static void DummyData(IEnumerable<Dictionary<string, object>> source, IEnumerable<Dictionary<string, object>> target)
        {
            // Dummy Data
            List<Dictionary<string, object>> lSource = source.ToList();
            List<Dictionary<string, object>> lTarget = target.ToList();

            for (int i = 0; i < 3; i++)
            {
                var sourceDic1 = new Dictionary<string, object>();
                sourceDic1.Add("Field1", "2017");
                sourceDic1.Add("Field2", "2018");
                sourceDic1.Add("Field3", "2019");
                sourceDic1.Add("Field4", "2018_E_" + i);

                lSource.Add(sourceDic1);                
            }


            for (int i = 2; i < 4; i++)
            {
                var targetDic2 = new Dictionary<string, object>();
                targetDic2.Add("Field1", "2017");
                targetDic2.Add("Field2", "2018");
                targetDic2.Add("Field3", "2019");
                targetDic2.Add("Field4", "2018_E_" + i);

                lTarget.Add(targetDic2);
            }


            // Results 
            var DoesNotExistInTarget = new List<Dictionary<string, object>>();
            var DoesNotExistInSource = new List<Dictionary<string, object>>();

            for (int i = 0; i < 2; i++)
            {
                var MissingDic1 = new Dictionary<string, object>();
                MissingDic1.Add("Field1", "2017");
                MissingDic1.Add("Field2", "2018");
                MissingDic1.Add("Field3", "2019");
                MissingDic1.Add("Field4", "2018_E_" + i);

                DoesNotExistInTarget.Add(MissingDic1);
            }

            for (int i = 3; i < 4; i++)
            {
                var MissingDic2 = new Dictionary<string, object>();
                MissingDic2.Add("Field1", "2017");
                MissingDic2.Add("Field2", "2018");
                MissingDic2.Add("Field3", "2019");
                MissingDic2.Add("Field4", "2018_E_" + i);

                DoesNotExistInSource.Add(MissingDic2);
            }    
        }        

Answer 1

OK I have updated now my answer according to your question update.

You will need one method that checks if dictionaries are equal.

The one below expects to have same keys in both dictionaries, as you need it, but if it happens to have different keys just add one check before the existing equality check.

I have implemented it as an extension method so it's easier to use it:

public static class DictionaryExtensions
{
    /// <summary>
    /// Expected to have 2 dictionaries with same keys
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="dict1"></param>
    /// <param name="dict2"></param>
    /// <returns></returns>
    public static bool IsEqual<T>(
        this Dictionary<string, T> dict1,
        Dictionary<string, T> dict2)
    {
        foreach (var keyValuePair in dict1)
        {
            if (!keyValuePair.Value.Equals(dict2[keyValuePair.Key]))
                return false;
        }

        return true;
    }
}

And then use this code inside your RecordCountResult method:

    public static IEnumerable<Dictionary<string, object>> RecordCountResults(
        IEnumerable<Dictionary<string, object>> source,
        IEnumerable<Dictionary<string, object>> target)
    {
        foreach (var sourceDictionary in source)
        {
            var existsInTarget = false;

            foreach (var targetDictionary in target)
            {
                if (sourceDictionary.IsEqual(targetDictionary))
                {
                    existsInTarget = true;
                    break;
                }    
            }

            if (!existsInTarget)
                yield return sourceDictionary;
        }
    }

The idea is that you have to loop first through source dictionaries and for each sourceDictionary check if it has the match in target list. If your sourceDictionary doesn't have a match in target it will be reported.

I didn't want to use Linq in RecordCountResults method because with foreach is more readable and easier to understand.

Also note that I'm using yield return instead of having temp list for a return candidates. If you prefer temp list, feel free to change it.