How to make fast GroupBy Query with Condition in Entity Framework

Question

I have a seriously performance problem with GROUP BY and WHERE in Entity Framework.

Here is an example query for the Northwind Database:

from order in Orders
join detail in OrderDetails on order.OrderID equals detail.OrderID
group detail by order.OrderDate into dateGroup
select new
{
    dateGroup.Key,
    Foo = dateGroup.Where(e => e.ProductID > 20).Sum(e => (decimal?)e.UnitPrice) ?? 0,
    Bar = dateGroup.Where(e => e.ProductID > 40).Sum(e => (decimal?)e.UnitPrice) ?? 0,
    Baz = dateGroup.Where(e => e.ProductID > 60).Sum(e => (decimal?)e.UnitPrice) ?? 0
}

This generates the following sql:

-- Region Parameters
DECLARE @p0 Int = 20
DECLARE @p1 Decimal(5,4) = 0
DECLARE @p2 Int = 40
DECLARE @p3 Decimal(5,4) = 0
DECLARE @p4 Int = 60
DECLARE @p5 Decimal(5,4) = 0
-- EndRegion
SELECT [t2].[OrderDate] AS [Key], COALESCE((
    SELECT SUM([t5].[value])
    FROM (
        SELECT [t4].[UnitPrice] AS [value], [t4].[ProductID], [t3].[OrderDate]
        FROM [Orders] AS [t3]
        INNER JOIN [Order Details] AS [t4] ON [t3].[OrderID] = [t4].[OrderID]
        ) AS [t5]
    WHERE ([t5].[ProductID] > @p0) AND ((([t2].[OrderDate] IS NULL) AND ([t5].[OrderDate] IS NULL)) OR (([t2].[OrderDate] IS NOT NULL) AND ([t5].[OrderDate] IS NOT NULL) AND ([t2].[OrderDate] = [t5].[OrderDate])))
    ),@p1) AS [Foo], COALESCE((
    SELECT SUM([t8].[value])
    FROM (
        SELECT [t7].[UnitPrice] AS [value], [t7].[ProductID], [t6].[OrderDate]
        FROM [Orders] AS [t6]
        INNER JOIN [Order Details] AS [t7] ON [t6].[OrderID] = [t7].[OrderID]
        ) AS [t8]
    WHERE ([t8].[ProductID] > @p2) AND ((([t2].[OrderDate] IS NULL) AND ([t8].[OrderDate] IS NULL)) OR (([t2].[OrderDate] IS NOT NULL) AND ([t8].[OrderDate] IS NOT NULL) AND ([t2].[OrderDate] = [t8].[OrderDate])))
    ),@p3) AS [Bar], COALESCE((
    SELECT SUM([t11].[value])
    FROM (
        SELECT [t10].[UnitPrice] AS [value], [t10].[ProductID], [t9].[OrderDate]
        FROM [Orders] AS [t9]
        INNER JOIN [Order Details] AS [t10] ON [t9].[OrderID] = [t10].[OrderID]
        ) AS [t11]
    WHERE ([t11].[ProductID] > @p4) AND ((([t2].[OrderDate] IS NULL) AND ([t11].[OrderDate] IS NULL)) OR (([t2].[OrderDate] IS NOT NULL) AND ([t11].[OrderDate] IS NOT NULL) AND ([t2].[OrderDate] = [t11].[OrderDate])))
    ),@p5) AS [Baz]
FROM (
    SELECT [t0].[OrderDate]
    FROM [Orders] AS [t0]
    INNER JOIN [Order Details] AS [t1] ON [t0].[OrderID] = [t1].[OrderID]
    GROUP BY [t0].[OrderDate]
    ) AS [t2]

As you can see, Foo, Bar and Baz is executed as a seperate subquery. Each subquery select and join again.

I have expected something that is more like this:

SELECT
  Orders.OrderDate,
  SUM(
    CASE
      WHEN [Order Details].ProductID > 20
      THEN [Order Details].UnitPrice
      ELSE 0
    END
  ) as Foo,
  SUM(
    CASE
      WHEN [Order Details].ProductID > 40
      THEN [Order Details].UnitPrice
      ELSE 0
    END
  ) as Bar,
  SUM(
    CASE
      WHEN [Order Details].ProductID > 60
      THEN [Order Details].UnitPrice
      ELSE 0
    END
  ) as Baz
FROM Orders
JOIN [Order Details] ON Orders.OrderID = [Order Details].OrderID
GROUP BY Orders.OrderDate

Ist it somehow possible to let the underlying linq provider generate better SQL for this scenario without using ctx.Database.SqlQuery<T>?

In my real scenario we are talking about 7 joins, nested group by and much more conditions. EF needs 180 seconds, SQL needs 3 seconds.

Answer 1

If you want to get a good SQL translation in LINQ to Entities (EF6), avoid using Where on the result of a GroupBy operator. Try using an equivalent conditional aggregates instead, e.g.:

Foo = dateGroup.Sum(e => e.ProductID > 20 ? (decimal?)e.UnitPrice : null) ?? 0,
Bar = dateGroup.Sum(e => e.ProductID > 40 ? (decimal?)e.UnitPrice : null) ?? 0,
Baz = dateGroup.Sum(e => e.ProductID > 60 ? (decimal?)e.UnitPrice : null) ?? 0