Need compare tables utility for SQL Server

Question

Command line or library "compare tables" utility for SQL server with comprehensive diff output to a file in .Net

I can't find anything like that. Commercial or free ( XSQL Lite is suitable for my case and ) tools show diffs in grids with possibility to export to CSV. Also they generate sync SQL scripts when run from command line. What I need is an output as a comprehensive report ( XML , HTML ) suitable for parsing so that I would be able to show similar diff grid in my application ( updated old/new values for each column , added - all values for row , deleted - all values for row and etc... ) .

Answer 1

Open Source DiffKit will do all of that except the .NET part.

www.diffkit.org

Answer 2

I;m assuming you want to compare data not schema and I'm assuming that the tables are in diff databases. I would not do it in SQL but load the data in generic .Net DataSet, and iterate thru Table object, columns and rows. This way one piece of code would work for any database table, no need for awkward dynamic SQL, however the downside is possible perf hit since you will need to load the data into a dataset - in this case just run the code on SQl server itself and dump the result files to a file share.

Answer 3

Since it sounds like you only want to show the diffs in your application, just write your own query, it isn't that hard, here is an example:

DECLARE @TableA table (RowID int, Col1 int, Col2 varchar(5), Col3 datetime)
DECLARE @TableB table (RowID int, Col1 int, Col2 varchar(5), Col3 datetime)
set nocount on
INSERT @TableA VALUES( 1,111,'AAA','1/1/2010')
INSERT @TableA VALUES( 2,222,'BBB','1/1/2010')
INSERT @TableA VALUES( 3,333,'CCC','1/1/2010')
INSERT @TableA VALUES( 4,444,'DDD','1/1/2010')
INSERT @TableA VALUES( 5,555,'EEE','1/1/2010')
INSERT @TableA VALUES( 6,666,'FFF','1/1/2010')
INSERT @TableA VALUES( 7,777,'GGG','1/1/2010')
INSERT @TableA VALUES( 9,888,'HHH','1/1/2010')
INSERT @TableA VALUES(10,111,'III','1/1/2010')

INSERT @TableB VALUES( 1,111,'AAA','1/1/2010')
INSERT @TableB VALUES( 3,333,'CCC','1/1/2010')
INSERT @TableB VALUES( 4,444,'DD' ,'1/1/2010')
INSERT @TableB VALUES( 5,555,'EEE','2/2/2010')
INSERT @TableB VALUES( 6,666,'FFF','1/1/2010')
INSERT @TableB VALUES( 7,777,'GGG','1/1/2010')
INSERT @TableB VALUES( 8,888,'ZZZ','1/1/2010')
INSERT @TableB VALUES( 9,888,'HHH','1/1/2010')
INSERT @TableB VALUES(10,111,'III','1/1/2010')
set nocount off

SELECT
    a.RowID, CASE WHEN b.RowID IS NULL THEN 'A' ELSE '' END AS RowsOnlyExistsIn
        ,a.Col1,b.Col1, CASE WHEN a.Col1=b.Col1 OR (COALESCE(a.Col1,b.Col1) IS NULL) THEN 'N' ELSE 'Y' END AS Col1Diff
        ,a.Col2,b.Col2, CASE WHEN a.Col2=b.Col2 OR (COALESCE(a.Col2,b.Col2) IS NULL) THEN 'N' ELSE 'Y' END AS Col2Diff
        ,a.Col3,b.Col3, CASE WHEN a.Col3=b.Col3 OR (COALESCE(a.Col3,b.Col3) IS NULL) THEN 'N' ELSE 'Y' END AS Col3Diff
    FROM @TableA                 a
        LEFT OUTER JOIN @TableB  b On a.RowID=b.RowID
UNION ALL
SELECT
    b.RowID, 'B' AS RowsOnlyExistsIn
        ,null,b.Col1, 'Y' AS Col1Diff
        ,null,b.Col2, 'Y' AS Col2Diff
        ,null,b.Col3, 'Y' AS Col3Diff
    FROM @TableB                 b
    WHERE b.RowID NOT IN (SELECT RowID FROM @TableA)
ORDER BY 1

OUTPUT:

RowID       RowsOnlyExistsIn Col1        Col1        Col1Diff Col2  Col2  Col2Diff Col3                    Col3                    Col3Diff
----------- ---------------- ----------- ----------- -------- ----- ----- -------- ----------------------- ----------------------- --------
1                            111         111         N        AAA   AAA   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
2           A                222         NULL        Y        BBB   NULL  Y        2010-01-01 00:00:00.000 NULL                    Y
3                            333         333         N        CCC   CCC   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
4                            444         444         N        DDD   DD    Y        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
5                            555         555         N        EEE   EEE   N        2010-01-01 00:00:00.000 2010-02-02 00:00:00.000 Y
6                            666         666         N        FFF   FFF   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
7                            777         777         N        GGG   GGG   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
8           B                NULL        888         Y        NULL  ZZZ   Y        NULL                    2010-01-01 00:00:00.000 Y
9                            888         888         N        HHH   HHH   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
10                           111         111         N        III   III   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N

(10 row(s) affected)

of course you would need to generate this dynamically so any two tables could be compared. This query will get you the columns of any tables:

SELECT
    *
    FROM INFORMATION_SCHEMA.COLUMNS
    WHERE 
        TABLE_CATALOG   ='database'
        AND TABLE_SCHEMA='dbo'
        AND TABLE_NAME  ='yourtable'
    ORDER BY ORDINAL_POSITION

here is a quick dynamic sql version (I attempt to pull in all the PK columns and use them to dynamically join the tables, but I only tested it on tables with 1 PK column):

set up for dynamic sql

CREATE TABLE TableA (RowID int primary key, Col1 int, Col2 varchar(5), Col3 datetime)
CREATE TABLE TableB (RowID int primary key, Col1 int, Col2 varchar(5), Col3 datetime)
set nocount on
INSERT TableA VALUES( 1,111,'AAA','1/1/2010')
INSERT TableA VALUES( 2,222,'BBB','1/1/2010')
INSERT TableA VALUES( 3,333,'CCC','1/1/2010')
INSERT TableA VALUES( 4,444,'DDD','1/1/2010')
INSERT TableA VALUES( 5,555,'EEE','1/1/2010')
INSERT TableA VALUES( 6,666,'FFF','1/1/2010')
INSERT TableA VALUES( 7,777,'GGG','1/1/2010')
INSERT TableA VALUES( 9,888,'HHH','1/1/2010')
INSERT TableA VALUES(10,111,'III','1/1/2010')

INSERT TableB VALUES( 1,111,'AAA','1/1/2010')
INSERT TableB VALUES( 3,333,'CCC','1/1/2010')
INSERT TableB VALUES( 4,444,'DD' ,'1/1/2010')
INSERT TableB VALUES( 5,555,'EEE','2/2/2010')
INSERT TableB VALUES( 6,666,'FFF','1/1/2010')
INSERT TableB VALUES( 7,777,'GGG','1/1/2010')
INSERT TableB VALUES( 8,888,'ZZZ','1/1/2010')
INSERT TableB VALUES( 9,888,'HHH','1/1/2010')
INSERT TableB VALUES(10,111,'III','1/1/2010')
set nocount off

dynamic sql

DECLARE @TableA   sysname
       ,@TableB   sysname
       ,@SQLa     varchar(max)
       ,@SQLb     varchar(max)
       ,@SQL      varchar(max)
SELECT @TableA='TableA'
      ,@TableB='TableB'
      ,@SQLa=NULL
      ,@SQLb=NULL

DECLARE @PKs      table (RowID int identity(1,1) primary key, PkColumn sysname)
DECLARE @index_id int
       ,@PK       sysname
       ,@i        int
SELECT @index_id=index_id from sys.indexes where object_id=OBJECT_ID(@TableA) AND is_primary_key=1

SELECT @PK=''
      ,@i=0
while (@PK is not null)
BEGIN
    SET @i=@i+1
    SELECT @PK = index_col(@TableA, @index_id, @i)
    IF @PK IS NULL BREAK
    INSERT INTO @PKs (PkColumn) VALUES (@PK)
END

SELECT @SQLa=''
      ,@SQLb=''''+@TableB+''' '
SELECT
    @SQLa=@SQLa+' ,a.'+COLUMN_NAME+',b.'+COLUMN_NAME+', CASE WHEN a.'+COLUMN_NAME+'=b.'+COLUMN_NAME+' OR (COALESCE(a.'+COLUMN_NAME+',b.'+COLUMN_NAME+') IS NULL) THEN ''N'' ELSE ''Y'' END AS '+COLUMN_NAME+'Diff '
    FROM INFORMATION_SCHEMA.COLUMNS
    WHERE TABLE_NAME  =@TableA
    ORDER BY ORDINAL_POSITION

SELECT
    @SQLb=@SQLb+' ,null,b.'+COLUMN_NAME+', ''Y'' AS '+COLUMN_NAME+'Diff'
    FROM INFORMATION_SCHEMA.COLUMNS
    WHERE TABLE_NAME  =@TableA
    ORDER BY ORDINAL_POSITION

SET @SQL='SELECT CASE WHEN b.'+(SELECT PkColumn FROM @PKs WHERE RowID=1)+' IS NULL THEN '''+@TableA+''' ELSE '''' END AS RowsOnlyExistsIn '
        +@SQLa
        +'FROM '+@TableA+' a LEFT OUTER JOIN '+@TableB+' b ON '

SELECT
    @SQL=@SQL+ CASE WHEN RowID!=1 THEN 'AND ' ELSE '' END +'a.'+PkColumn+'=b.'+PkColumn+' '
    FROM @PKs

SET @SQL=@SQL+' UNION ALL SELECT '
        +@SQLb
        +' FROM '+@TableB+' b LEFT OUTER JOIN '+@TableA+' A ON '

SELECT
    @SQL=@SQL+ CASE WHEN RowID!=1 THEN 'AND ' ELSE '' END +'b.'+PkColumn+'=a.'+PkColumn+' '
    FROM @PKs

SET @SQL=@SQL+'WHERE a.'+(SELECT PkColumn FROM @PKs WHERE RowID=1)+' IS NULL ORDER BY 2,3'

EXEC(@SQL)

output:

RowsOnlyExistsIn RowID       RowID       RowIDDiff Col1        Col1        Col1Diff Col2  Col2  Col2Diff Col3                    Col3                    Col3Diff
---------------- ----------- ----------- --------- ----------- ----------- -------- ----- ----- -------- ----------------------- ----------------------- --------
TableB           NULL        8           Y         NULL        888         Y        NULL  ZZZ   Y        NULL                    2010-01-01 00:00:00.000 Y
                 1           1           N         111         111         N        AAA   AAA   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
TableA           2           NULL        Y         222         NULL        Y        BBB   NULL  Y        2010-01-01 00:00:00.000 NULL                    Y
                 3           3           N         333         333         N        CCC   CCC   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
                 4           4           N         444         444         N        DDD   DD    Y        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
                 5           5           N         555         555         N        EEE   EEE   N        2010-01-01 00:00:00.000 2010-02-02 00:00:00.000 Y
                 6           6           N         666         666         N        FFF   FFF   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
                 7           7           N         777         777         N        GGG   GGG   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
                 9           9           N         888         888         N        HHH   HHH   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N
                 10          10          N         111         111         N        III   III   N        2010-01-01 00:00:00.000 2010-01-01 00:00:00.000 N

(10 row(s) affected)