CODEWITHSUNDEEP
c#.netlinqentity-frameworkef-code-first
Jaanus
Jaanus

Reputation: 16531

EF LINQ ToList is very slow

I am using ASP NET MVC 4.5 and EF6, code first migrations.

I have this code, which takes about 6 seconds.

var filtered = _repository.Requests.Where(r => some conditions); // this is fast, conditions match only 8 items
var list = filtered.ToList(); // this takes 6 seconds, has 8 items inside

I thought that this is because of relations, it must build them inside memory, but that is not the case, because even when I return 0 fields, it is still as slow.

var filtered = _repository.Requests.Where(r => some conditions).Select(e => new {}); // this is fast, conditions match only 8 items
var list = filtered.ToList(); // this takes still around 5-6 seconds, has 8 items inside

Now the Requests table is quite complex, lots of relations and has ~16k items. On the other hand, the filtered list should only contain proxies to 8 items.

Why is ToList() method so slow? I actually think the problem is not in ToList() method, but probably EF issue, or bad design problem.

Anyone has had experience with anything like this?

EDIT:

These are the conditions:

_repository.Requests.Where(r => ids.Any(a => a == r.Student.Id) && r.StartDate <= cycle.EndDate && r.EndDate >= cycle.StartDate)

So basically, I can checking if Student id is in my id list and checking if dates match.

Upvotes: 3

Views: 14672

Answers (5)

Keith
Keith

Reputation: 738

Your first line of code only returns an IQueryable. This is a representation of a query that you want to run not the result of the query. The query itself is only runs on the databse when you call .ToList() on your IQueryable, because its the first point that you have actually asked for data.

Your adjustment to add the .Select only adds to the existing IQueryable query definition. It doesnt change what conditions have to execute. You have essentially changed the following, where you get back 8 records:

select * from Requests where [some conditions];

to something like:

select '' from Requests where [some conditions];

You will still have to perform the full query with the conditions giving you 8 records, but for each one, you only asked for an empty string, so you get back 8 empty strings.

The long and the short of this is that any performance problem you are having is coming from your "some conditions". Without seeing them, its is difficult to know. But I have seen people in the past add .Where clauses inside a loop, before calling .ToList() and inadvertently creating a massively complicated query.

Upvotes: 1

Hamid Pourjam
Hamid Pourjam

Reputation: 20744

In addition to Maarten's answer I think the problem is about two different situation

  1. some condition is complex and results in complex and heavy joins or query in your database

  2. some condition is filtering on a column which does not have an index and this cause the full table scan and make your query slow.

I suggest start monitoring the query generated by Entity Framework, it's very simple, you just need to set Log function of your context and see the results,

using (var context = new MyContext())
{
    context.Database.Log = Console.Write;

    // Your code here...
}

if you see something strange in generated query try to make it better by breaking it in parts, some times Entity Framework generated queries are not so good.

if the query is okay then the problem lies in your database (assuming no network problem).

run your query with an SQL profiler and check what's wrong.

UPDATE

I suggest you to:

  1. add index for StartDate and EndDate Column in your table (one for each, not one for both)

Upvotes: 3

esolCrusador
esolCrusador

Reputation: 51

Jaanus. The most likely reason of this issue is complecity of generated SQL query by entity framework. I guess that your filter condition contains some check of other tables.

Try to check generated query by "SQL Server Profiler". And then copy this query to "Management Studio" and check "Estimated execution plan". As a rule "Management Studio" generatd index recomendation for your query try to follow these recomendations.

Upvotes: 0

Maarten
Maarten

Reputation: 22945

Your filtered variable contains a query which is a question, and it doesn't contain the answer. If you request the answer by calling .ToList(), that is when the query is executed. And that is the reason why it is slow, because only when you call .ToList() is the query executed by your database.

It is called Deferred execution. A google might give you some more information about it.

If you show some of your conditions, we might be able to say why it is slow.

Upvotes: 7

Max Brodin
Max Brodin

Reputation: 3938

ToList executes the query against DB, while first line is not.

Can you show some conditions code here? To increase the performance you need to optimize query/create indexes on the DB tables.

Upvotes: 2

Related Questions