CODEWITHSUNDEEP
c#performancelinqlinq-to-entitiesienumerable
Anarion
Anarion

Reputation: 2534

C#, filtering IEnumerable with multiple .Where() statements, performance hits

I have an IEnumerable, and I'm applying multiple filters to it via .Where expression. My code looks something like this

public List<MyObject> FilteringFunction(List<MyObject> listToFilter, List<Filter> filters)
{
    // A dirty way to have an enumerable instead of List
    var enumerableToFilter = listToFilter.Where(x => true);

    foeach(var filter in filters)
    {
        enumerableToFilter = enumerableToFilter.Where(x => x.Value.Contains(filter.Value));
    }

    return enumerableToFilter.ToList();

}

Am I going to iterate through my collection only once? (As I would have only one database call with LINQ to SQL)

Upvotes: 0

Views: 7115

Answers (3)

Kelson Ball
Kelson Ball

Reputation: 983

Enumerables defer execution until you iterate over them, and multiple filters are applied on a single iteration through the collection. Combining with other linq statements may force early enumeration, I haven't tested every combination. This would only be a problem with very large data sets or on low spec performance critical systems.

Here is an example using Visual Studios c# interactive

> class Item
. {
.     private int _number;
.     public int Number
.     {
.         get { Console.WriteLine($"Got number {_number}"); return _number; }
.         set { _number = value; }
.     }
. }
> 
> IEnumerable<Item> items = new List<Item>() { 
.     new Item { Number = 1 },
.     new Item { Number = 2 },
.     new Item { Number = 3 },
.     new Item { Number = 4 },
.     new Item { Number = 5 },
.     new Item { Number = 6 }
. };
> 
> var filteredItems = items.Where(item => item.Number > 3).Where(item => item.Number % 2 == 0);
> 
> var listedItems = filteredItems.ToList();
Got number 1
Got number 2
Got number 3
Got number 4
Got number 4
Got number 5
Got number 5
Got number 6
Got number 6
>

Note that 1, 2, and 3 are filtered out and the second filter method is not called on them. 4, 5, and 6 all pass the first filter so both filters are applied.

Key Point: Note that the filtering doesn't actually happen until the enumerable is read to a list. You will be able to continue appending filters until you enumerate the result to a list.

Upvotes: 5

Tarek Abo ELkheir
Tarek Abo ELkheir

Reputation: 1331

First of all the result will be empty for sure sine your filtering the same property with the different values, and let say you corrected this, answering on your question, for iqueryable it will call the database just once, for Enumerable object is the same , as the iteration will happen just when you iterate the object with foreach or calling GetEnumerator , in your case you are not doing that

Upvotes: 1

Thomas Levesque
Thomas Levesque

Reputation: 292465

The overhead could become noticeable if you have many filters, due to the many delegate creations and invocations and many calls to Where. That being said, you should probably run performance tests to determine if it's really an issue.

You could refactor the code to test all the filters in the same predicate:

public List<MyObject> FilteringFunction(List<MyObject> listToFilter, List<Filter> filters)
{
    Func<MyObject, bool> predicate =
        x =>
            filters.All(f => x.Value.Contains(f.Value));

    return listToFilter.Where(predicate).ToList();
}

PS: regarding this:

// A dirty way to have an enumerable instead of List

There's a cleaner way:

var enumerableToFilter = listToFilter as IEnumerable<MyObject>;

or even:

var enumerableToFilter = listToFilter.AsEnumerable();

Upvotes: 0

Related Questions