SQL: Reporting on data by date

Question

I have a table I am trying to generate reports from. Its basically a log of when something breaks (goes down), and then gets fixed.

Table schema and some example data is below. To illustrate:
1 row is inserted when it goes down
1 row is inserted when it comes back up.

What I am trying to do is report on various aspects, things like:
Amount of downtime in a given day / week / month
Number of times its gone down in a given day / week / month.

Ideally in a way that would easily be exported to excel or something similar to be graphed.

I'm having trouble coming up with any kind of queries to get this info.

I have this one for example:

SELECT [Name], datepart(day,[Inserted]), count([SystemDown])     
FROM [DownTimeLog]
WHERE [SystemDown]=1
GROUP BY [Name],datepart(day,[Inserted]) 

which gives me the number of times the system has gone down per day, which is a good starting point.

But i'm trying to come up with a way of showing the total time its been down, but I am drawing a blank. Some days for example may be 0, sometimes it may go down a few times, so trying to sum the time difference between 2 corresponding rows is provinh tough.

CREATE TABLE [dbo].[DownTimeLog]
(
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [Name] VARCHAR(30) NOT NULL,
    [SystemDown] BIT NOT NULL,
    [Inserted] DATETIME NOT NULL
)

INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 14 2011  1:49:58:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 14 2011  2:49:58:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  1:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  2:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  4:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  5:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 17 2011  1:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 17 2011  3:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011 10:00:00:000AM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011 11:00:00:000AM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  1:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  3:30:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  4:00:00:000PM')
INSERT INTO [DownTimeLog] ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  8:00:00:000PM')

So for example using the data above I'd like to pull data back that was something like:

System1 | June 14 2011 | 1 hour | 1 occurence
System1 | June 15 2011 | 2 hours | 2 occurence's
System1 | June 16 2011 | 0 hours | 0 occurence's
System1 | June 17 2011 | 2 hours | 1 occurence
System1 | June 18 2011 | 7.5 hours | 3 occurence's

I hope someone can give me a method of doing what I am trying to do.

-- Edit:

Thanks all for some great answers. Helped me out a tonne. I thought my sql was pretty string, never heard of a cross apply though - guess i need to go back to school!

Cheers!

Answer 1

The following will produce your exact output including missing days. Note, this code is based on Mikael Eriksson's answer.

CREATE TABLE #DownTimeLog
(
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [Name] VARCHAR(30) NOT NULL,
    [SystemDown] BIT NOT NULL,
    [Inserted] DATETIME NOT NULL
)

INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 14 2011  1:49:58:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 14 2011  2:49:58:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  1:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  2:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 15 2011  4:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 15 2011  5:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 17 2011  1:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 17 2011  3:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011 10:00:00:000AM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011 11:00:00:000AM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  1:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  3:30:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',1,'Jun 18 2011  4:00:00:000PM')
INSERT INTO #DownTimeLog ([Name],[SystemDown],[Inserted])VALUES('System1',0,'Jun 18 2011  8:00:00:000PM')

CREATE TABLE #DownTimeLogModified (Name nvarchar(512), [Date] nvarchar(512), DownTime int,  Occurrences int)

INSERT INTO #DownTimeLogModified (Name, [Date], DownTime,  Occurrences)
select D1.Name,
       CONVERT(VARCHAR(12), dateadd(d, datediff(d, 0, D1.Inserted), 0), 107) as [Date],
       sum(datediff(mi, D1.Inserted, D2.Inserted)) as DownTime,
       count(*) as Occurrences
from #DownTimeLog as D1
  cross apply ( select top 1 Name,
                             Inserted
                from #DownTimeLog
                where SystemDown = 0 and
                      Name = D1.Name and
                      Inserted > D1.Inserted 
                order by Inserted             
              ) as D2
where D1.SystemDown = 1
group by D1.Name, dateadd(d, datediff(d, 0, D1.Inserted), 0)

DECLARE @startdate datetime,@enddate datetime
select @startdate = MIN(Inserted) FROM #DownTimeLog
select @enddate = MAX(Inserted) FROM #DownTimeLog

;WITH DateIntervalsCTE AS
        (
        SELECT 1 i,@startdate AS Date
        UNION ALL
        SELECT i + 1, DATEADD(day, i, @startdate )
        FROM DateIntervalsCTE 
        WHERE DATEADD(day, i, @startdate ) <= @enddate
        )

SELECT CASE WHEN a.Name is null THEN 'No System downtime' ELSE a.Name END as Name,CONVERT(VARCHAR(12), b.Date, 107) AS Date,CASE WHEN a.DownTime is null THEN 0 ELSE a.DownTime END AS DownTime,CASE WHEN a.Occurrences is null THEN 0 ELSE a.Occurrences END AS Occurrences 
FROM DateIntervalsCTE b
LEFT JOIN #DownTimeLogModified a ON a.Date = CONVERT(VARCHAR(12), b.Date, 107)

DROP TABLE #DownTimeLog
DROP TABLE #DownTimeLogModified

Output

Name                           Date                    DownTime    Occurrences
------------------------------ ----------------------- ----------- -----------
System1                        Jun 14, 2011                60           1
System1                        Jun 15, 2011               120           2
No system downtime             Jun 16, 2011                 0           0
System1                        Jun 17, 2011               120           1
System1                        Jun 18, 2011               450           3

Answer 2

WITH
  enhanced_log
AS
(
SELECT
  [start].Name                                              AS [Name],
  DATEADD(DAY, DATEDIFF(DAY, 0, [start].Inserted), 0)       AS [Date],
  [start].Inserted                                          AS [Start],
  [finish].Inserted                                         AS [Finish]
FROM
  DownTimeLog  AS [start]
OUTER APPLY
  (SELECT TOP 1 * FROM DownTimeLog WHERE name = [start].name AND Inserted > [start].Inserted ORDER BY Inserted ASC) AS [finish]
WHERE
  [start].SystemDown = 1

UNION ALL

SELECT
  [start].Name,
  [Calendar].date,
  [Calendar].date,
  [finish].Inserted
FROM
  calendar
CROSS JOIN
  system
CROSS APPLY
  (SELECT TOP 1 * FROM DownTimeLog WHERE name = [system].name AND Inserted < [calendar].date ORDER BY Inserted ASC) AS [start]
OUTER APPLY
  (SELECT TOP 1 * FROM DownTimeLog WHERE name = [system].name AND Inserted > [start].Inserted ORDER BY Inserted ASC) AS [finish]
WHERE
  [start].SystemDown = 1
)

SELECT
  Name,
  Date,
  SUM(DATEDIFF(MINUTE, Start, CASE WHEN Finish < Date + 1 THEN Finish ELSE Date + 1 END)) AS Duration,
  COUNT(*) AS Instances
FROM
  enhanced_log
WHERE
  Start <> CASE WHEN Finish < Date + 1 THEN Finish ELSE Date + 1 END
GROUP BY
  Name,
  Date

Answer 3

I think if you break these into two pseudo-tables you will be able to do more. Try this:

SELECT
    *
FROM
    ( SELECT
        *
      FROM
        [DownTimeLog] AS down
      WHERE
        [SystemDown] = 1
    ) down
    INNER JOIN ( SELECT
                    *
                 FROM
                    [DownTimeLog]
                 WHERE
                    SystemDown = 0
               ) up
    ON down.id = ( up.id - 1 ) and down.name = up.name
    ORDER BY down.[Inserted], up.inserted

You can then do all kinds of calculations on down records vs. the subsequent up records.

EDIT: Of course and @Dems points out this assumes IDs are sequential. If you would rather use dates as the filter do this:

SELECT
    *
FROM
    ( SELECT
        *
      FROM
        [DownTimeLog] AS down
      WHERE
        [SystemDown] = 1
    ) down
    CROSS APPLY ( SELECT TOP 1
                    *
                 FROM
                    [DownTimeLog] up
                 WHERE
                    SystemDown = 0

                    AND up.INSERTED > down.INSERTED
                    AND up.NAME = down.name
                    ORDER BY up.inserted 
               ) up
    ORDER BY down.[Inserted], up.inserted

Here is a more detailed example of what can be done:

SELECT
        [Down_Id]
    ,   [Down_Name]
    ,   [Down_SystemDown]
    ,   [Down_Inserted]
    ,   [Up_Id]
    ,   [Up_Name]
    ,   [Up_SystemDown]
    ,   [Up_Inserted]
    ,   CAST(DATEDIFF(mi,Down_Inserted,Up_Inserted) AS DECIMAL)/60  AS Hours_Down
    ,   DATEDIFF(mi,Down_Inserted,Up_Inserted)  AS Minutes_Down
FROM
    ( SELECT
        [ID] AS Down_Id
      , [Name] AS Down_Name
      , [SystemDown] AS Down_SystemDown
      , [Inserted] AS Down_Inserted
      FROM
        [DownTimeLog] AS down
      WHERE
        [SystemDown] = 1
    ) down
    CROSS APPLY ( SELECT TOP 1
                    [ID] AS Up_Id
                  , [Name] AS Up_Name
                  , [SystemDown] AS Up_SystemDown
                  , [Inserted] AS Up_Inserted
                  FROM
                    [DownTimeLog] up
                  WHERE
                    SystemDown = 0
                    AND up.[Inserted] > down.Down_Inserted
                    AND up.NAME = down.Down_name
                  ORDER BY
                    up.Inserted
                ) up
ORDER BY
    down.[Down_Inserted]
,   up.up_inserted

Answer 4

Something like this might work for you.

select D1.Name,
       dateadd(d, datediff(d, 0, D1.Inserted), 0) as [Date],
       sum(datediff(mi, D1.Inserted, D2.Inserted)) as DownTime,
       count(*) as Occurrences
from DownTimeLog as D1
  cross apply ( select top 1 Name,
                             Inserted
                from DownTimeLog
                where Name = D1.Name and
                      Inserted > D1.Inserted 
                order by Inserted             
              ) as D2
where D1.SystemDown = 1
group by D1.Name, dateadd(d, datediff(d, 0, D1.Inserted), 0)

Result:

Name                           Date                    DownTime    Occurrences
------------------------------ ----------------------- ----------- -----------
System1                        2011-06-14 00:00:00.000 60          1
System1                        2011-06-15 00:00:00.000 120         2
System1                        2011-06-17 00:00:00.000 120         1
System1                        2011-06-18 00:00:00.000 450         3

You won't get a row for days with 0 occurrences. And if you have downtime over midnight all time will be counted on the day of the down event. DownTime is in minutes.