CODEWITHSUNDEEP
c#linqlookup
Suresh
Suresh

Reputation: 219

LINQ tolookup using C#

I have a huge list of the point datatype and an equal size of a double list. The size could be around 1000000. Both lists comes from a different class.

List<Point> XYPair;  E.g. { (10,10),(10,10).......(20,20),(20,20).....}
List<double> ZValue; E.g. { 1.5, 1.6, .............7.8,8.7......}

I need to plot unique XY pairs as X and Y co-ordinate. But to apply markercolor for the single XY, I need to lookup all the corresponding ZValue for that single XY and apply statistics. Index of XYPair and ZValue matches. Is it possible to use LINQ tolookup to achieve this efficiently rather than getting an outofmemory expection error by doing like below?

Currently I am doing it like this:

GroupedPlotValues = XYLocation
    .Select(bv => bv.ElementAt(0))
    .Select((p, i) => new { Item1 = p, Item2 = i })
    .GroupBy(tp => tp.Item1, tp => tp.Item2)
    .ToDictionary(gr => gr.Key, gr => gr.ToList());


foreach (var item in GroupedPlotValues)
{
    var matched = item.Value.Intersect(MatchingIndexes).ToList();
    if (matched.Count != 0)
    {
        chart1.Series[0].Points.AddXY(item.Key.X, item.Key.Y);
        var singleZGroup = item.Value.Select(y => ZColumn[y]).ToList();
        ApplyStatistics(singleZGroup);
    }
}

Upvotes: 3

Views: 6450

Answers (2)

wageoghe
wageoghe

Reputation: 27608

What if you did something like this?

Gather your points...

var XY = new List<Point>()
{
  { new Point(0, 0) },
  { new Point(10, 20) },
  { new Point(15, 5)},
  { new Point(0,0)},
  { new Point(10,20)}
};

Gather your Z values...

var Z = new List<double>() { 0, 10, 20, 30, 40 };

Use Zip to build a new list containing (Point, Z) pairs, then convert the resulting sequence to a lookup (similar to a dictionary of groups), keyed by the Point value.

var lookup = XY.Zip(Z, (xy, z) => new { Point = xy, Z = z }).ToLookup(k => k.Point, v => v.Z);

I don't know what the performance/memory characteristics will be, but I think that it does give the functionality you are after.

Some articles I found helpful in formulating this answer:

Implementing the Zip Operator in .NET 3.5 provides an implementation of Zip that you can use...

As I state in my comment below, the performance seems fine on my machine, so my machine might be faster, my data might not be distributed the same as yours, or what I consider "fine" you might consider "slow". Having said that, I am adding some code that might be performe better than your version (or might not). If this doesn't help enough, I don't know what else to add. Also, in your original question you say that you run out of memory. Does that still happen with my suggested code?

My rewrite of your code:

//Gather your position data
var XY = new List<Point>();
{
  { new Point(0, 0) },
  { new Point(10, 20) },
  { new Point(15, 5)},
  { new Point(0,0)},
  { new Point(10,20)}
};

//Gather your Z values ..
var Z = new List<double>() { 0, 10, 20, 30, 40 };

//Build the lookup
var lookup = XY.Zip(Z, (xy, z) => new { Point = xy, Z = z }).ToLookup(k => k.Point, v => v.Z);

//Process...
//foreach unique XY (the Key of the lookup)
//  Check to see if the XY is in a set of MatchingIndexes.  If not, continue.
//  Add the unique point to the chart series.
//  Get the Z values that correspond to the key (no need for ToList unless ApplyStatistics needs something more specialized than IEnumerable).
//  Apply the Z values by calling ApplyStatistics
//
foreach (g in lookup.Select(g => g.Key))
{
  var matched = MatchingIndexes.Select(i => i == g);
  if (!matched.Any()) continue;
  chart1.Series[0].Points.AddXY(g.X, g.Y);
  var singleZGroup = lookup[g];
  ApplyStatistics(singleZGroup);
}

Note that I have not tested the processing code above. I hope that it works, and I hope that it does the equivalent work that you were doing in your original code. I don't have any particular expectations that this will be "fast" or not. The main goal in my code is to minimize the number of times that data is copied (typically by calls to ToList).

Good luck, I hope this helped.

Upvotes: 2

Massimo Zerbini
Massimo Zerbini

Reputation: 3191

Why do you need LINQ? Just use a loop:

// Prepare the dictionary
Dictionary<Key, List<double>> dic = new Dictionary<Key, List<double>>();
for (int i =0; i< XYPair.Count; i++){
    // Create the key
    // Put ZValue[i] values in dictionary list element
}

// Use the Dictionary:

// Loop around dic keys
// If matched, apply statistics

Upvotes: 2

Related Questions