Optimizing CLUSTERED INDEX for use with JOIN

Question

table optin_channel_1 (for each 'channel' there's a dedicated table)

CREATE TABLE [dbo].[optin_channel_1](
    [key_id] [bigint] NOT NULL,
    [valid_to] [datetime] NOT NULL,
    [valid_from] [datetime] NOT NULL,
    [key_type_id] [int] NOT NULL,
    [optin_flag] [tinyint] NOT NULL,
    [source_proc_id] [int] NOT NULL,
    [date_inserted] [datetime] NOT NULL
) ON [PRIMARY]

CREATE CLUSTERED INDEX [ix_id] ON [dbo].[optin_channel_1] 
(
    [key_type_id] ASC,
    [key_id] ASC,
    [valid_to] ASC,
    [valid_from] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]

table profile_conns

CREATE TABLE [dbo].[profile_conns](
    [profile_key_id] [bigint] NOT NULL,
    [valid_to] [datetime] NOT NULL,
    [valid_from] [datetime] NOT NULL,
    [conn_key_id] [bigint] NOT NULL,
    [conn_key_type_id] [int] NOT NULL,
    [conn_type_id] [int] NOT NULL,
    [source_proc_id] [int] NOT NULL,
    [date_inserted] [datetime] NOT NULL
) ON [PRIMARY]

CREATE CLUSTERED INDEX [ix_id] ON [dbo].[profile_conns] 
(
    [profile_key_id] ASC,
    [conn_key_type_id] ASC,
    [conn_key_id] ASC,
    [valid_to] ASC,
    [valid_from] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]

table lu_channel_conns

CREATE TABLE [dbo].[lu_channel_conns](
    [channel_id] [int] NOT NULL,
    [conn_type_id] [int] NOT NULL,
 CONSTRAINT [PK_lu_channel_conns] PRIMARY KEY CLUSTERED 
(
    [channel_id] ASC,
    [conn_type_id] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

table lu_conn_type

CREATE TABLE [dbo].[lu_conn_type](
    [conn_type_id] [int] NOT NULL,
    [default_key_type_id] [int] NOT NULL,
    [master_key_type_id] [int] NOT NULL,
    [date_inserted] [datetime] NOT NULL,
 CONSTRAINT [PK_lu_conns] PRIMARY KEY CLUSTERED 
(
    [conn_type_id] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

view v_source_proc_id_by_group_id

SELECT DISTINCT x.source_proc_id, x.source_proc_group_id
FROM lu_source_proc x INNER JOIN lu_source_proc_group y ON x.source_proc_group_id = y.group_id

There's a dynamic SQL statement going to be executed:

SET @sql_str='SELECT @ret=MAX(o.optin_flag)
    FROM optin_channel_'+CAST(@channel_id AS NVARCHAR(100))+' o
    INNER HASH JOIN dbo.v_source_proc_id_by_group_id y ON o.source_proc_id=y.source_proc_id AND y.source_proc_group_id=@source_proc_group_id
    INNER HASH JOIN profile_conns z ON z.profile_key_id=cast(@profile_key_id AS NVARCHAR(100)) AND z.conn_key_type_id=o.key_type_id AND z.conn_key_id=o.[key_id] AND z.valid_to=''01.01.3000''
    INNER HASH JOIN lu_channel_conns x ON x.channel_id=@channel_id AND z.conn_type_id=x.conn_type_id
    INNER HASH JOIN lu_conn_type ct ON ct.conn_type_id=x.conn_type_id AND ct.default_key_type_id=o.key_type_id'
SET @param='@channel_id INT, @profile_key_id INT, @source_proc_group_id INT, @ret NVARCHAR(400) OUTPUT'
EXEC sp_executesql @sql_str,@param,@channel_id,@profile_key_id,@source_proc_group_id,@ret OUTPUT

I.e. this gives:

SELECT @ret=MAX(o.optin_flag) AS optin_flag
FROM optin_channel_1 o
INNER HASH JOIN dbo.v_source_proc_id_by_group_id y 
    ON o.source_proc_id=y.source_proc_id 
    AND y.source_proc_group_id=5
INNER HASH JOIN profile_conns z 
    ON z.profile_key_id=1 
    AND z.conn_key_type_id=o.key_type_id 
    AND z.conn_key_id=o.[key_id] 
    AND z.valid_to='01.01.3000'
INNER HASH JOIN lu_channel_conns x 
    ON x.channel_id=1 
    AND z.conn_type_id=x.conn_type_id
INNER HASH JOIN lu_conn_type ct 
    ON ct.conn_type_id=x.conn_type_id 
    AND ct.default_key_type_id=o.key_type_id

These tables are used for an optin database. optin_flag could be 0 or 1. With the last statement I want to get a 1 as optin_flag from optin_channel_1 for the given channel_id=1 for user with profile_key_id=1, when optin was inserted into database by process belonging to source_proc_group_id=5. I hope this is enough to comprehend what's going on.

Is this the best way to use the CLUSTERED INDEX'es? Or would it be better to remove profile_key_id from index on profile_conns and put z.profile_key_id=1 in a WHERE clause?

May be there's a much better way for optimizing this select (changes in database schema is not possible, only changes on indexes and modifing statement).

Answer 1

Without knowing the size of the tables and the sort of data stored in it them it is difficult to gauge.

Assuming optin_channel_1 has a lot of data and profile_cons has a lot of data I would try the following:

• Clustered index on optin_channel_1(key_id) or key_type_id depending on which field has the most distinct values. (since you don't have a covering index)
• Clustered index on profile_conns (cons_key_id) or cons_key_type_id depending on what you have chosen in optin_channel_1
• etc...

Basically, if your table profile_conns table has not much data, I would put the clustered index on the most fragmented "filter" field (I suspect profile_key_id). If the table has a lot of data I would aim for a hash/merge join and match the clustered index with the clustered index of the optin_channel_1 table.

I would also rewrite the query as such:

SELECT @ret = MAX(o.optin_flag) AS optin_flag
  FROM optin_channel_1 o
  JOIN dbo.v_source_proc_id_by_group_id y 
    ON o.source_proc_id = y.source_proc_id  
  JOIN profile_conns z 
    ON z.conn_key_type_id = o.key_type_id 
   AND z.conn_key_id = o.[key_id] 
  JOIN lu_channel_conns x 
    ON z.conn_type_id = x.conn_type_id
  JOIN lu_conn_type ct 
    ON ct.conn_type_id = x.conn_type_id 
   AND ct.default_key_type_id=o.key_type_id 
 WHERE y.source_proc_group_id = 5
   AND z.profile_key_id = 1 
   AND x.channel_id = 1 
   AND z.valid_to = '01.01.3000'

The query changed this way because:

• Putting the filter conditions in the where clause shows you what are relevant fields to aim for a hash/merge join
• Putting join hints is rarely a good idea. It is very hard to beat the query governor to determine the best query plan. A bad plan usually indicates you have an issue with your indexes/statistics.

So as summary:

• small table joined to big table ==> go for nested loops & focus your clustered index on the "filter" field in the small table & the join field in the big table.
• big table joined to big table => go for hash/merge join and put the clustered index on the matching field on both sides
• multi-field indexes usually only a good idea when they are "covering", this means all the fields you query are included in the index. (or are included with the include() clause)