CODEWITHSUNDEEP
c#sqllistgenerics
user2966445
user2966445

Reputation: 1344

Reasonable number of List <string> elements?

I know there are limitations to the number of items that can be added to a List (basically int.MaxValue), but what is a reasonable number to use in an everyday scenario - 100k, 500k, 1 million?

I'm using a DataReader and adding one string at a time to a List, then processing the elements as a batch. I want to ensure I'm not pushing memory limits too much by making the batches too large.

Upvotes: 2

Views: 145

Answers (2)

Richard Schneider
Richard Schneider

Reputation: 35477

Another approach is to return an IEnumerable (not List) and use the yield keyword. This will not consume extra memory for a list.

public IEnumerable<string> MyBigList(DataReader reader)
{
    while (reader.Read())
    {
       yield return (string) reader["SomeField"];
    }
}

In the old days this was called a co-routine, in C# its referred to as an iterator. See MSDN for more information on iterators and yield.

To process the string(s)

foreach (var s in MyBigList(reader))
{
   // do something with s
}

Upvotes: 6

Yeldar Kurmangaliyev
Yeldar Kurmangaliyev

Reputation: 34189

It depends.

The main idea behind the size of count of strings is that every character takes 2 bytes of memory by default.

Some considerations:

  • If every string in a list is a full text of a classic novel, then 100 items can certainly bring some problems, but if this list is a dictionary of correctly spelled words of average 7-8 characters long, then 100k strings shouldn't be a problem since it's just a couple megabytes.
  • How many of lists are you going to have? If it is a single instance, then it takes X bytes of memory. But if it is an ASP.NET application and you create a List for every user request, then it is going to grow exponentially.
  • If you write an application for a powerful server station, then multiple-gigabyte cache collection can be a reasonable solution, but if you write a desktop client application with low system requirements, then you cannot afford it.
  • If you make many scan operations (search), add and remove items, save it or load from file system, then every operation is taking O(N) operations. Probably, you would like to change used collection type according to your problem or use a database.
  • If you process items one by one and you can avoid working with it as with collection, then you don't need List at all.
  • etc.

In most cases, "What is the reasonable number of List items" question is not as critical as
"Is this a reasonable case for using List at all?".
In-memory dynamic-size plain collections (i.e., List) can either be useful or useless in different usage scenarios, and we can't know which one is your case (especially, without your code).

Upvotes: 2

Related Questions