CODEWITHSUNDEEP
mysqlsqlgroup-by
user1387312
user1387312

Reputation: 45

Time interval SQL query with MySQL

I've got a table in a database that stores log data by time. For one day there can be a million rows in the db. The times are not at any regular interval. It has several indexes, including the time. What I want to do is build a query that will return a set of rows with one row per time interval. For example, I could do a query to return 1 row every 15 minutes for a day. This would return 24*60=96 rows. Each row returned would actually be the nearest row in the db prior to the interval requested (since the data in the database will not equal the requested interval).

I am at a loss for how to do it. I can't just query all rows for a particular set of indexes and time interval, as it would load more than a gigabyte of data into memory, which is too slow. Is there any efficient way to do this using SQL. I'm using a MySQL database. I would be open to changing the table indexes/etc...

TIME

11:58
12:03
12:07
12:09
12:22
12:27
12:33
12:38
12:43
12:49
12:55

If I wanted to query this for a 15 minute interval from 12:00 to 1:00, I'd get back:

11:58 (nearest 12:00)
12:09 (nearest 12:15)
12:27 (nearest 12:30)
12:43 (nearest 12:45)
12:55 (nearest 1:00)

If it makes it any easier, I can also store the time as a number (i.e. ms since 1970). In the above query, this would then be an interval of 900000 ms.

Upvotes: 3

Views: 9790

Answers (3)

Jason Goemaat
Jason Goemaat

Reputation: 29234

I think using functions is pretty easy and I haven't noticed big performance implications, although a cursor would probably preform better depending on how many rows there are between times.

CREATE TABLE TEST_TIMES (EventTime datetime)
-- skipping INSERTS of your times

CREATE FUNCTION fn_MyTimes ( @StartTime datetime, @EndTime datetime, @Minutes int )
    RETURNS @TimeTable TABLE (TimeValue datetime)
AS BEGIN
    DECLARE @CurrentTime datetime
    SET @CurrentTime = @StartTime
    WHILE @CurrentTime <= @EndTime
    BEGIN
        INSERT INTO @TimeTable VALUES (@CurrentTime)
        SET @CurrentTime = DATEADD(minute, @Minutes, @CurrentTime)
    END
    RETURN
END

CREATE FUNCTION fn_ClosestTime ( @CheckTime datetime )
    RETURNS datetime
AS BEGIN
    DECLARE @LowerTime datetime, @HigherTime datetime

    SELECT @LowerTime = MAX(EventTime)
    FROM TEST_TIMES
    WHERE EventTime <= @CheckTime

    SELECT @HigherTime = MAX(EventTime)
    FROM TEST_TIMES
    WHERE EventTime >= @CheckTime

    IF @LowerTime IS NULL RETURN @HigherTime -- both null?  then null
    IF @HigherTime IS NULL RETURN @LowerTime

    IF DATEDIFF(ms, @LowerTime, @CheckTime) < DATEDIFF(ms, @CheckTime, @HigherTime)
        RETURN @LowerTime
    RETURN @HigherTime
END

SELECT TimeValue, dbo.fn_ClosestTime(TimeValue) as ClosestTime
FROM fn_MyTimes('2012-05-17 12:00', '2012-05-17 13:00', 15)

Results:

TimeValue               ClosestTime
----------------------- -----------------------
2012-05-17 12:00:00.000 2012-05-17 11:58:00.000
2012-05-17 12:15:00.000 2012-05-17 12:09:00.000
2012-05-17 12:30:00.000 2012-05-17 12:27:00.000
2012-05-17 12:45:00.000 2012-05-17 12:43:00.000
2012-05-17 13:00:00.000 2012-05-17 12:55:00.000

Upvotes: 1

Andrew
Andrew

Reputation: 4624

So, I had thought something like:

SELECT 
  MIN(timeValue)
FROM e
GROUP BY (to_seconds(timeValue) - (to_seconds(timeValue) % (60 * 5)))

..would do it for you, but this only returns the MIN(timeValue) over the whole table. It works if the seconds rounded to the nearest 5 min is in its own col.

See SQL Fiddle

Edit per Andiry, this works: ( http://sqlfiddle.com/#!2/bb870/6 )

SELECT MIN(t)
FROM e
GROUP BY to_seconds(t) DIV (60 * 5)

But this just gives one row: ( http://sqlfiddle.com/#!2/bb870/7 )

SELECT MIN(t)
FROM e
GROUP BY to_seconds(t) - (to_seconds(t) % (60 * 5))

Anyone know why?

Upvotes: 4

Travesty3
Travesty3

Reputation: 14489

I can't think of a good way to do it all in one query. Perhaps someone else can think of a better way, but perhaps you could use something like this:

$startTime = mktime(12, 0);
$endTime = mktime(13, 0);
$queries = array();
for ($i = $startTime; $i <= $endTime; $i += 900)
    $queries[] = "SELECT MAX(timeValue) FROM table1 WHERE timeValue < '". date("G:i", $i) ."'";

$query = implode("\nUNION\n", $queries);

I just realized that this assumes that you are using PHP. If you are not, then just use the resulting query, which will look like:

SELECT MAX(timeValue) FROM table1 WHERE timeValue < '12:00'
UNION
SELECT MAX(timeValue) FROM table1 WHERE timeValue < '12:15'
UNION
SELECT MAX(timeValue) FROM table1 WHERE timeValue < '12:30'
UNION
SELECT MAX(timeValue) FROM table1 WHERE timeValue < '12:45'
UNION
SELECT MAX(timeValue) FROM table1 WHERE timeValue < '13:00'

Not sure if the < comparison will work 100% correctly with these string values, but I definitely think it would be a good idea to switch them to unix timestamps (or ms since 1970, if you need that much granularity). I have found it's always easier to work with integer values for date/time instead of strings.

Upvotes: 0

Related Questions