CODEWITHSUNDEEP
perl
ungalnanban
ungalnanban

Reputation: 10327

Date Range Problem

I have a log file which has first few characters of every line as a timestamp.

2010-06-01 04:56:02,802 DEBUG {Thread-27} Some text message

2010-06-01 04:56:02,802 DEBUG {Thread-27} Some text message

2010-06-01 04:56:02,802 DEBUG {Thread-27} Some text message

2010-06-01 04:56:02,802 DEBUG {Thread-27} Some text message

2010-06-01 05:22:02,802 DEBUG {Thread-27} Some text message

2010-06-01 05:22:02,802 DEBUG {Thread-27} Some text message

2010-06-01 05:22:02,802 DEBUG {Thread-27} Some text message

2010-06-01 05:22:02,802 DEBUG {Thread-27} Some text message

2010-06-01 06:43:02,802 INFO {Thread-27} Some text message

2010-06-01 06:43:02,803 INFO {Thread-27} Some text message

2010-06-01 06:43:02,804 INFO {Thread-27} Some text message

2010-06-01 06:43:02,804 INFO {Thread-27} Some text message

2010-06-01 06:43:02,809 DEBUG {Thread-27} Some text message

2010-06-01 06:43:02,809 DEBUG {Thread-27} Some text message

2010-06-01 06:43:02,809 DEBUG {Thread-27} Some text message

2010-06-01 07:08:02,809 DEBUG {Thread-27} Some text message

2010-06-01 07:08:02,809 DEBUG {Thread-27} Some text message

My aim to find all such lines which have the timestamp of 1 hr before the current time.

How can this be achieved?

Upvotes: 3

Views: 381

Answers (3)

David Wall
David Wall

Reputation: 51

Does the order that the lines are output matter? If you don't mind having them with the most recent first, you might consider using File::ReadBackwards. Keep reading backwards until a line is more than one hour old, then stop. If you want them in a particular order, you could store them in an array and print them however you want. (This assumes that it's a more-or-less standard log file with the most recent entries at the end of the file.)

Upvotes: 0

Zaid
Zaid

Reputation: 37146

The DateTime module is well-suited to the needs of this problem:

use strict;
use warnings;
use DateTime;

my $oneHourAgo = DateTime->now()->subtract( hours => 1 );
my $threshold  = join ' ', $oneHourAgo->ymd, $oneHourAgo->hms;  # Time as string

open my $logFile, '<', 'logfile.txt';

while (my $log = <$logFile>) {

    chomp $log;
    my ($time) = split /,/, $log;       # Gets current log's time

    print $log if $time ge $threshold;  # String-compares log's time to threshold
}

close $logFile;

Upvotes: 4

Aryabhatta
Aryabhatta

Reputation:

Since the timestamps are going to be sorted, you can try a kind of Binary Search with a twist.

Since mostly the lines won't be of same length, you could just seek to a certain offset, look for the newlines (or whichever line terminator you have) which appears before and after (OR after and the one after that), you get a candidate line. Now compare the date on the line to the one you are looking for and decide whether to seek again, or just look around in the neighbourhood of this line.

In determing what offset to seek to next, you could try using something similar to what Interpolation Search does, i.e. decide the offset based on the difference between the time of the line you got and the time you are searching for.

This should be much faster than linear search.

For an example to do binary search in files using perl: http://perl.plover.com/yak/lightweight-db/materials/slides/slide024.html

Upvotes: 1

Related Questions