CODEWITHSUNDEEP
c#hashset
mqpasta
mqpasta

Reputation: 960

HashSet for unique strings up to millions of records?

I am generating millions of unique string. I am using HashSet<> for this purpose. I have to store the results in text file periodically.

I noticed my code takes too much time at following lines:

for (int i = lastEndIndex; i < storage.Count; i++)
        {
            sb.AppendLine(storage.ElementAt(i));
        }

The lastEndIndex is required in order to store results generated after last save. Is there any other better mechanism for this?

Upvotes: 0

Views: 284

Answers (1)

MarcinJuraszek
MarcinJuraszek

Reputation: 125630

ElementAt() has to iterate from the beginning of the collection every time it's called. That's why it's so inefficient.

I would suggest another approach - save results generated after last save into a list, and save those results into file from there, not from HastSet itself:

latest list:

var latest = new List<string>();

Adding elements:

if(storage.Add(newElement))
{
    latest.Add(newElement);
}

Saving latest to file:

foreach(var item in latest)
{
    sb.AppendLine(item);
}
latest.Clear();

Upvotes: 2

Related Questions