CODEWITHSUNDEEP
c#sql-serverentity-framework
Zholen
Zholen

Reputation: 1790

Normalizing SQL table while using Entity Framework

Let me start by saying I am not a SQL expert, novice is appropriate

I have a log table which currently has 10 million rows:

Logs( Id, FKId, Source, Message )

I am wondering if it would be beneficial to Normalize my table for Source and Message. Of the 10m rows there are only 200,000 DISTINCT Source's and 1 million Distinct Messages. To normalize this data I assume I would need two more tables. a Source with Id and Text, and a Message with Id and Text (or perhaps they could be the same table?)

I am using Entity Framework currently to insert rows. I am wondering how Entity Framework would handle this and if I would get the performance boost I am looking for, specifically when trying to do "Message LIKE '%error%'", or just in general by making the whole thing smaller thus less paging.

I assume there is no way that EF will automatically figure out for me if the Message of the new insert is unique so I would have to figure that out before hand or do the insert via a Stored Procedure instead. If i'm doing 2 million inserts a day to this table how does calling a SP via EF change in terms of performance vs 

context.Logs.Add(log)
context.Save();

Upvotes: 0

Views: 1697

Answers (1)

Dai
Dai

Reputation: 155270

I assume your C# code would have a method signature like this:

public void AddLogMessage(Int32 foreignKeyId, String source, String message)

If you modify your DB design to be like this:

Logs( LogId, FKId, SourceId, MessageId )
Sources( SourceId, Value )
Messages( MessageId, Value )
-- with all appropriate FK constraints on Logs->Sources and Logs->Messages

...then you'll find yourself writing more C#/Linq than is necessary to lookup the source and message text and put them into the record (in addition to using more DB chatter). Pragmatically this is best done with a Stored Procedure that you would add to your DBContext as a Function Import:

CREATE PROCEDURE AddLogMessage
    @fkId int,
    @source nvarchar(255),
    @message nvarchar(1024)
AS
BEGIN
    SET NOCOUNT ON

    BEGIN TRANSACTION

    DECLARE @sourceId int = SELECT SourceId FROM Sources WHERE Value = @source
    IF @sourceId IS NULL
    BEGIN
        INSERT INTO Sources ( Value ) VALUES ( @source )
        SET @sourceId = SELECT SCOPE_IDENTITY()
    END

    DECLARE @messageId int = SELECT MessageId FROM Messages WHERE Value = @message
    IF @messageId IS NULL
    BEGIN
        INSERT INTO Messages ( Value ) VALUES ( @message)
        SET @messageId = SELECT SCOPE_IDENTITY()
    END

    INSERT INTO Logs ( @fkId, @sourceId, @messageId )

    COMMIT TRANSACTION

END

After you've imported it into your DBContext, just call it like any other function:

public void AddLogMessage(Int32 foreignKeyId, String source, String message) {
    this.dbContext.AddLogMessage( foreignkeyId, source, message );
}

Upvotes: 2

Related Questions