CODEWITHSUNDEEP
c#
Brady
Brady

Reputation:

Removing duplicates from a list collection

Hope someone can help me. I am using c# and I am somewhat new to it. I am loading a text file into my app and splitting the data on "," I am reading part of string into a <list> when the data is read there is a lot of duplicates that vary depending on the txt file that I load. Can someone tell me how to check the and remove any and all duplicates that come up. There is no way of knowing what duplicates will show up as there is infinite possibilities that it could be.

Thanks for the help in advance

Upvotes: 3

Views: 5145

Answers (4)

itowlson
itowlson

Reputation: 74842

If you are targeting .NET 3.5, use the Distinct extension method:

var deduplicated = list.Distinct();

Upvotes: 7

C. Ross
C. Ross

Reputation: 31878

A simple/dirty example follows:

public List<string> RemoveDuplicates(List<string> listWithDups)
{
   cleanList = new List<string>();
   foreach (string s in listWithDups)
   {
      if (!cleanList.Contains(s))
         cleanList.Add(s);
   }
   return cleanList;
}

As a warning: String.Split on very large strings can cause consume huge amounts of memory and cause exceptions.

Upvotes: 2

ChrisW
ChrisW

Reputation: 56123

If you load the strings into a Set instead of into a List then duplicates are discarded automatically.

Upvotes: 4

Alex Fort
Alex Fort

Reputation: 18819

Here's an article with some examples and explanations in C#. Basically, you just keep track of the uniques, and check each element.

Alex

Upvotes: 1

Related Questions