Linq query taking a long time to query a table with 6000 records

Question

I am having trouble with the performance of a query I am trying to write. I have a table with about 6000 records within it.

Currently its taking about 15 seconds to run on my development machine. A win8 machine with 32gb ram quadcore running vs2012 & sql2012. So its not my machine its my bad code.

    public IEnumerable<Customer> GetByStoreIdAndContainingName(Guid storeId, string containing)
    {
        using (var context = new Entities())
        {
            var store = context.Stores.FirstOrDefault(b => b.StoreId == storeId);
            if (store == null) return null;
            var business = store.Business;
            var consumers = new List<Consumer>();


            consumers =
                business.ConsumerIdentities.Select(ci => ci.Consumer)
                        .Distinct()
                        .Where(x => x.FirstName.ToLower().Contains(containing.ToLower()))
                        .ToList();

The layout of the database tables are

Business

BusinessId
Name
etc

StoreId

StoreId
StoreName
BusinessId

Consumer

ConsumerId
FirstName
LastName

ConsumerIdentities

BusinessId
ConsumerIdentityType
ConsumerIdentityValue

Can anyone see any obvious things I am doing wrong that would be taking so long to return the query results?

Turning on SQL Profiler was scary. The first query made was to select everything from ConsumerIdentity table where the business Id matched. Great that's perfect then gets the business table table.

However then seems to make a call for every single record like

exec sp_executesql N'SELECT 
[Extent1].[ConsumerId] AS [ConsumerId], 
[Extent1].[UserID] AS [UserID], 
[Extent1].[FirstName] AS [FirstName], 
[Extent1].[LastName] AS [LastName], 
[Extent1].[IsMale] AS [IsMale], 
[Extent1].[DateOfBirth] AS [DateOfBirth], 
FROM [dbo].[Consumer] AS [Extent1]
WHERE [Extent1].[ConsumerId] = @EntityKeyValue1',N'@EntityKeyValue1     uniqueidentifier',@EntityKeyValue1='952ED7B8-2123-49E2-BAE3-69FBD713BACB'

So it looks like my where statement isn't getting applied

Answer 1

There are couple of ways to optimize that may decrease the process time (that other guys told) but any way you're query is doing a search process on a database with 6000 records. But it should do it faster than 15sec for 6000 records.

It may because of your memory cache. I recommend clear your VS and system cache then try again. (you can use C-Cleaner app for system cache)

You can use Async in ADO too boots up. It has a great effect in decreasing the process time. (You can do it with IDataReadables and IDataReaders that support async)

Answer 2

I would start with Customer (if the properties are present) because that saves a Distinct in the first place and it should do everything in one SQL statement:

from c in Customer
where c.FirstName.ToLower().Contains(containing.ToLower())
      && c.ConsumerIdentity.Business.StoreId == storeId
select c

This does not take away the where on c.FirstName.ToLower(). Such constructs always hit performance because they are not sargable (they eliminate any index). EF currently has no tooling to do case-insensitive search (it does not accept the Contains overload with an IEqualityComparer), so technically you can't avoid this. But chances are that the database collation is case insensitive, so x.FirstName.Contains(containing.ToLower() might give the same result, with index.

Answer 3

Does it help if you do the Select after the Where?

        consumers =
            business.ConsumerIdentities
                    .Where(x => x.FirstName.ToLower().Contains(containing.ToLower()))
                    .Distinct()
                    .Select(ci => ci.Consumer)
                    .ToList();

Answer 4

I just see one thing that can be improved:

• Use the Distinct after the Where (Distinct is a O(n log n) Where in O(n))

      consumers =
          business.ConsumerIdentities.Select(ci => ci.Consumer)
                  .Where(x => x.FirstName.ToLower().Contains(containing.ToLower()))
                  .Distinct()
                  .ToList();
  

But I doubt your perfomance issue comes from the LINQ query (6000 records should be extremely fast in any case).