CODEWITHSUNDEEP
c#unique-values
user34537
user34537

Reputation:

Efficient list of unique strings C#

What is the most efficient way to store a list of strings ignoring any duplicates? I was thinking a dictionary may be best inserting strings by writing dict[str] = false; and enumerating through the keys as a list. Is that a good solution?

Upvotes: 104

Views: 95010

Answers (7)

Dave Hollingsworth
Dave Hollingsworth

Reputation: 171

You could also use Linq as in:

using System.Linq;

var items = new List<string>() { "one", "one", "two", "one", "two", "zero" };

List<string> distinctItems = items.Distinct().ToList();

Upvotes: 15

Alexey Solonets
Alexey Solonets

Reputation: 817

Here is another solution without using the HashSet.

var items = new List<string>() { "one", "one", "two", "one", "two", "zero" };
var uniqueItems = items.Where((item, index) => items.IndexOf(item) == index);

It was adopted from this thread: javascript - Unique values in an array

Test:

using FluentAssertions;

uniqueItems.Count().Should().Be(3);
uniqueItems.Should().BeEquivalentTo("one", "two", "zero");

Performance test for List, HashSet and SortedSet. 1 million iterations:

List: 564 ms
HashSet: 487 ms
SortedSet: 1932 ms

Test source code (gist)

Upvotes: 2

Perpetualcoder
Perpetualcoder

Reputation: 13591

You can look to do something like this 

var hash = new HashSet<string>();
var collectionWithDup = new []{"one","one","two","one","two","zero"}; 

// No need to check for duplicates as the Add method
// will only add it if it doesn't exist already
foreach (var str in collectionWithDup)
    hash.Add(str);

Upvotes: 33

Priyang
Priyang

Reputation: 386

Use HashSet, no need to check .Contains() , just add your items in list and if its duplicate it will not add it.

   HashSet<int> uniqueList = new HashSet<int>();
   uniqueList.Add(1); // List has values 1
   uniqueList.Add(2);  // List has values 1,2
   uniqueList.Add(1);  // List has values 1,2
   Console.WriteLine(uniqueList.Count); // it will return 2

Upvotes: 10

scone
scone

Reputation: 576

I'm not sure if this counts as a good answer, but when faced with the need for a unique set that maintains insertion order, I compromised with a HashSet and a List side-by-side. In this case, whenever you add to the set, do the following:

if(hashSet.Add(item))
    orderList.Add(item);

When removing items, make sure to remove them from both. Thus, as long as you can be sure that nothing else added items to the list, you'll have an insertion-ordered unique set!

Upvotes: 15

AndrewB
AndrewB

Reputation: 972

This is not part of the the system namespace but have used the Iesi.Collections from http://www.codeproject.com/KB/recipes/sets.aspx with NHibernate. It has support for hashed set along with sorted set, dictionary set, and so on. Since it has been used with NHibernate it has been used extensively and very stable. This also does not require .Net 3.5

Upvotes: 2

JP Alioto
JP Alioto

Reputation: 45127

If you are using .NET 3.5, the HashSet should work for you.

The HashSet<(Of <(T>)>) class provides high performance set operations. A set is a collection that contains no duplicate elements, and whose elements are in no particular order.

Upvotes: 125

Related Questions