LINQ and memory allocation

Question

In this question under the term LINQ I mean LINQ to objects.

1. Which of the LINQ methods allocate new memory and which are not?

2. For example, does Select(x => x) allocate new memory?

3. If there are different methods in LINQ - some allocate memory and another ones aren't - have I check and consider every method in a LINQ chain in the terms of memory efficiency? So using LINQ is not just-simple-stupid adding whatsoever methods in a LINQ chain? So I need to constantly keep in mind every LINQ method "feature" before using it in a chain? In another words, is there no single rule or pattern for using LINQ (like "LINQ always does/doesn't allocate new memory") in the terms of memory efficiency?

Answer 1

The MSDN for Enumerable.Select specifies that it "This method is implemented by using deferred execution. The immediate return value is an object that stores all the information that is required to perform the action." Hence, it allocates at least memory for the lambda (x=>x) and for the reference to the collection it is invoked on. However, this takes memory in the order of the complexity of the query and not in the order of the size of the collection and should therefore be mostly negligible.

Memory is only allocated in the order of the size of the collection when the result of the query is iterated. As Pareek's link shows, this usually happens once per query. In LINQ and Deferred Execution (even though it relates to LINQ to SQL), gives a good statement on this: If a LINQ method returns something different than IEnumerable it has to iterate the collection and may therefore allocate new memory (e.g. ToList(), ToArray()) but doesn't have to (e.g. Count()).

Hence, if you end up having some sort of collection in the end which is no IEnumerable, the query will allocate memory in the order of the size of this resulting collection.