CODEWITHSUNDEEP
c#design-patterns
user2547359
user2547359

Reputation: 61

Memory efficient way to store many small objects

I have a simple Person class with 4 strings and integer.

public class Person
{
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public int Age { get; set; }
        public string PostalCode { get; set; }
}

We need to store a large number of these in memory. The collection needs to be searchable by any field. Items will be added and deleted as part the life cycle.

The Flyweight pattern doesn't seem work because there isn't a ton of duplicate values at the object, only at the field level. What pattern or strategy would work best to limit memory overhead and perform well?

Upvotes: 5

Views: 4012

Answers (2)

Haney
Haney

Reputation: 34922

A combination of things might work here... First, it's best left as a reference type so that you're not copying structs everywhere when searching. Use string.Intern(string) to reduce memory usage on duplicate first and last names as well as postal codes... Finally use Dictionary<TKey, TValue> to index these entries by value... Perhaps the TKey is string in the case of FirstName and TValue' is List<Person> so that you can look up the people by said string... This is called an inverted index: http://en.wikipedia.org/wiki/Inverted_index - an alternative to Dictionary is to implement your own Tree or Trie structure such as a Prefix trie... You trade O(log n) speed for less memory than the O(1) dictionary.

In terms of storing a LOT of them in memory, it depends what a lot is... but ultimately you want to have enough memory to handle them all... Or begin to expand out to distributed systems to share the objects such as a MapReduce pattern, or "paging" onto disk.

Upvotes: 0

Henk Holterman
Henk Holterman

Reputation: 273854

We need to store a large number of these in memory.

Then an array Person[] would be the leanest way but a List<Person> would be close and much easier to work with. Just make sure to minimize re-allocation by using the Capacity parameter.

The collection needs to be searchable by any field

Easy, .Where (p => p.FirstName == value).
Speeding it up with Dictionaries will cost memory.

Upvotes: 2

Related Questions