CODEWITHSUNDEEP
sqlsql-serverrelational-database
isekaijin
isekaijin

Reputation: 19772

SQL optimization question

Let's start with the scenario defined in my previous question.

Now I want to create a query that generates the list of Foos F1 and the count of Foos F2 that are distinct than F1 but are nevertheless associated to the same Bar or Baz F1 is associated to:

SELECT    F1.*,
          CASE
            WHEN F1.Bar_ID IS NOT NULL THEN
              ISNULL(Bar.LotNumber + '-', '') + Bar.ItemNumber
            WHEN F2.Baz_ID IS NOT NULL THEN
              ISNULL(Baz.Color + ' ', '') + Baz.Type
          END AS 'Ba?Description',
          (SELECT COUNT(*)
           FROM   Foo F2
           WHERE  F2.Bar_ID = F1.Bar_ID
           OR     F2.Baz_ID = F1.Baz_ID) - 1 AS FooCount
FROM      Foo F1
LEFT JOIN Bar ON Bar.Bar_ID = F1.Bar_ID
LEFT JOIN Baz ON Baz.Baz_ID = F1.Baz_ID

What worries me is efficiency. I must admit I know nothing regarding how SQL Server generates execution plans from SQL sentences, but common sense tells me that the subquery would be executed once for each row in the main query, i.e., once for each value of F1.Foo_ID. This is clearly not efficient.

An alternative is that does not run into this problem is...

SELECT    F1.*,
          CASE
            WHEN F1.Bar_ID IS NOT NULL THEN
              ISNULL(Bar.LotNumber + '-', '') + Bar.ItemNumber
            WHEN F2.Baz_ID IS NOT NULL THEN
              ISNULL(Baz.Color + ' ', '') + Baz.Type
          END AS 'Ba?Description',
          COUNT(*) - 1 AS FooCount
FROM      Foo F1
LEFT JOIN Bar    ON Bar.Bar_ID = F1.Bar_ID
LEFT JOIN Baz    ON Baz.Baz_ID = F1.Baz_ID
LEFT JOIN Foo F2 ON F2 .Bar_ID = F1.Bar_ID
                 OR F2 .Baz_ID = F1.Baz_ID
GROUP BY  F1.Foo_ID, F1.SomeFooField, F1.SomeOtherField, ...,
          CASE
            WHEN F1.Bar_ID IS NOT NULL THEN
              ISNULL(Bar.LotNumber + '-', '') + Bar.ItemNumber
            WHEN F2.Baz_ID IS NOT NULL THEN
              ISNULL(Baz.Color + ' ', '') + Baz.Type
          END

But this is even worse, since it runs into a bigger problem that is related to the fact that SQL databases are not true relational databases. If SQL databases were truly relational, then SQL engines would be able to infer that the value of every field that is not affected by an aggregate function is uniquely determined by F1.Foo_ID. Thus, GROUP BY F1.Foo_ID should be sufficient to produce the desired result. But SQL still forces me to explicitly GROUP BY every field not affected by an aggregate function. The result? Inefficiency.

A third alternative that does not run into any of the two previous problems is...

SELECT    Foo.*,
          CASE
            WHEN Foo.Bar_ID IS NOT NULL THEN
              ISNULL(Bar.LotNumber + '-', '') + Bar.ItemNumber
            WHEN Foo.Baz_ID IS NOT NULL THEN
              ISNULL(Baz.Color + ' ', '') + Baz.Type
          END AS 'Ba?Description',
          ISNULL(Temp.FooCount, 0) AS FooCount
FROM      Foo
LEFT JOIN Bar ON Bar.Bar_ID = Foo.Bar_ID
LEFT JOIN Baz ON Baz.Baz_ID = Foo.Baz_ID
LEFT JOIN (SELECT   F1.Foo_ID, COUNT(*) - 1 AS FooCount
           FROM     Foo F1
           JOIN     Foo F2 ON F2.Bar_ID = F1.Bar_ID
                           OR F2.Baz_ID = F1.Baz_ID
           GROUP BY F1.Foo_ID) Temp ON Temp.Foo_ID = Foo.Foo_ID

But this has the disadvantage of requiring the instantiation of three copies of Foo in memory, not just two.

How should I structure my query to produce the desired result in the most efficient way possible?

Upvotes: 0

Views: 149

Answers (1)

boes
boes

Reputation: 2855

I agree with the comments stating that you can only find out by trying. In your other post you say that you have no test data available. So my guess is you don't know how to generate test data. I'll show you.

I assume the following tables exist:

create table Bar (
    Bar_ID int not null primary key,
    LotNumber varchar(10),
    ItemNumber varchar(10)
)

create table Baz (
    Baz_ID int not null primary key,
    Color varchar(10),
    Type varchar(10)
)

create table Foo (
    Foo_ID int not null primary key,
    Bar_ID int null references Bar,
    Baz_ID int null references Baz,
    SomeFooField varchar(10),
    SomeOtherFooField varchar(10)
)

Now populate Bar with test data:

insert into Bar (Bar_ID) values (0)
insert into Bar (Bar_ID) select Bar_ID + 1 from Bar
insert into Bar (Bar_ID) select Bar_ID + 2 from Bar
insert into Bar (Bar_ID) select Bar_ID + 4 from Bar
insert into Bar (Bar_ID) select Bar_ID + 8 from Bar
insert into Bar (Bar_ID) select Bar_ID + 16 from Bar
insert into Bar (Bar_ID) select Bar_ID + 32 from Bar
insert into Bar (Bar_ID) select Bar_ID + 64 from Bar
-- etc. 

update Bar set 
    LotNumber = 'LN_' + convert(varchar(10), Bar_ID), 
    ItemNumber = 'IN_' + convert(varchar(10), Bar_ID)

Populate Baz:

insert into Baz (Baz_ID) values (0)
insert into Baz (Baz_ID) select Baz_ID + 1 from Baz
insert into Baz (Baz_ID) select Baz_ID + 2 from Baz
insert into Baz (Baz_ID) select Baz_ID + 4 from Baz
insert into Baz (Baz_ID) select Baz_ID + 8 from Baz
insert into Baz (Baz_ID) select Baz_ID + 16 from Baz
insert into Baz (Baz_ID) select Baz_ID + 32 from Baz
-- etc

update Baz set 
    Color = 'C_' + convert(varchar(10), Baz_ID), 
    Type = 'T_' + convert(varchar(10), Baz_ID)

and put some data in Foo

insert into Foo (Foo_ID) values (0)
insert into Foo (Foo_ID) select Foo_ID + 1 from Foo
insert into Foo (Foo_ID) select Foo_ID + 2 from Foo
insert into Foo (Foo_ID) select Foo_ID + 4 from Foo
insert into Foo (Foo_ID) select Foo_ID + 8 from Foo
insert into Foo (Foo_ID) select Foo_ID + 16 from Foo
insert into Foo (Foo_ID) select Foo_ID + 32 from Foo
insert into Foo (Foo_ID) select Foo_ID + 64 from Foo
insert into Foo (Foo_ID) select Foo_ID + 128 from Foo
insert into Foo (Foo_ID) select Foo_ID + 256 from Foo
-- etc...

update Foo set 
    SomeFooField = 'SFF_' + convert(varchar(10), Foo_ID), 
    SomeOtherFooField = 'SOFF_' + convert(varchar(10), Foo_ID)

update Foo set Bar_ID = Bar.Bar_ID
    from Bar 
    where Foo_ID % 128 = Bar.Bar_ID
        and Foo_ID % 3 = 0;

update Foo set Baz_ID = Baz.Baz_ID
    from Baz 
    where Foo_ID % 64 = Baz.Baz_ID
        and Foo_ID % 3 <> 0

Before you run the queries and test their execution speed, make sure you have some indexes created:

create index Foo_Baz on Foo(Baz_ID)
create index Foo_Bar on Foo(Bar_ID)

Now you can test your queries. I suggest you try this one too:

select F.Foo_id, F.*,
    isNull(R.barDescription, Z.bazDescription) as 'Ba?Description', 
    isnull(R.fooCount, Z.fooCount) - 1 as fooCount
from Foo F
left join (
    select F.Bar_ID, 
        ISNULL(Bar.LotNumber + '-', '') + Bar.ItemNumber as 'BarDescription',
        count(F.Foo_id) as FooCount 
    from Foo F, Bar
    where F.Bar_id = Bar.Bar_id
    group by F.Bar_id, Bar.LotNumber, Bar.ItemNumber
) R on F.Bar_ID = R.Bar_ID
left join (
    select F.Baz_ID,
        ISNULL(Baz.Color + '-', '') + Baz.Type as 'BazDescription',
        count(F.Foo_id) as FooCount
    from Foo F, Baz
    where F.Baz_id = Baz.Baz_id
    group by F.Baz_id, Baz.Color, Baz.Type
) Z on F.Baz_ID = Z.Baz_ID

In my old version of SQL Query Analyzer there is an option to 'Display the generated execution plan'. Your version will have that option too probably. It shows that the above query will run faster than the 3 queries you suggested. But that is theory! So fill your tables with as much data as you think it will have in the production system and try.

Upvotes: 1

Related Questions