Perl How to find specific Start and end line from the log file?

Question

The file is structured like this:

<28 Jan 2014 00:08:24>00001 startmessage1
<28 Jan 2014 00:08:24>00001 msg1
<28 Jan 2014 00:08:24>00001 msg1
<28 Jan 2014 00:08:24>00001 startmessage2
<28 Jan 2014 00:08:25>00001 msg2
<28 Jan 2014 00:08:25>00001 msg2
<28 Jan 2014 00:08:25>00001 msg2
<28 Jan 2014 00:08:25>00001 msg2
<28 Jan 2014 00:08:25>00001 msg2
<28 Jan 2014 00:08:26>00001 msg2
<28 Jan 2014 00:08:26>00001 Endmessage2
<28 Jan 2014 00:08:26>00001 msg1
<28 Jan 2014 00:08:26>00001 msg1
<28 Jan 2014 00:08:27>00001 startmessage3
<28 Jan 2014 00:08:27>00001 msg3
<28 Jan 2014 00:08:28>00001 msg1
<28 Jan 2014 00:08:28>00001 msg1
<28 Jan 2014 00:08:30>00001 Endmessage1
<28 Jan 2014 00:08:31>00001 msg3
<28 Jan 2014 00:08:31>00001 msg3
<28 Jan 2014 00:08:31>00001 msg3
<28 Jan 2014 00:08:31>00001 Endmessage3

Date and time, ID, log message... have to find the start message and corresponding end message to culculate the timing.

output should be

28 Jan 2014 00:08:24 |startmessage1 |Endmessage1 |00001| 6 seconds
28 Jan 2014 00:08:24 |startmessage2 |Endmessage2 |00001| 2 seconds

My code:

my @startmsg= ("startmessage1","startmessage2","startmessage3");
my @endmsg= ("Endmessage1","Endmessage2","Endmessage3");
print 'THREAD ID,LOG DETAILS'."\n";

     foreach my $filename (glob("$dir/*.out"))
        {
          open(my $fh, "<", $filename) or die "Could not open '$filename'\n";

                while ($line = <$fh>)
                {
                        my $i=2;
                        chomp($line);

                         if($line=~ $startmsg[$i])
                        {
                                chomp ($line);
                                my $threadid=$line;
                                $threadid =~ s!(\d*)\s*.*!$1!;
                                my $startdetails=$line;
                                $startdetails =~ s!(\d*)\s*(.*)!$2!;
                                chomp($threadid);
                                print "$threadid,$startdetails \n";
                                my $flag = 'False';

                               OUTER:  while ($line = <$fh>)
                                {
                                        if ($line=~ m/$threadid/)
                                         if($line=~  $endmsg[$i])
                                                {
                                                chomp ($line);
                                                my $enddetails=$line;
                                                $enddetails =~ s!(\d*)\s*(.*)!$2!;
                                                print "$threadid,$enddetails\n";
                                                $flag='True';
                                                last OUTER;
                                                }

                                        }
                                }
                        }

                }
                 close($fh);
        }

"Im able to get only one start and end message in this program. Also i tried with assigning all the staring into one variable ( all start messages in one variable and all end messages in one variable) .. the problem is after start message1 , end message 2 got captured and calculated the wrong time difference. please help me to fix the issue"

Note : Start message and end messages are different. i just gave the examples. Also the same line I'm scraping few more things

Please help me

Answer 1

Using DateTime to handle the date-time handling. Save the messages in a hash:

#!/usr/bin/perl
use warnings;
use strict;
use feature qw{ say };

use DateTime;

my %message;

sub month2num {
    my $name = shift;
    my %months;
    @months{ qw(Jan Feb Mar Apr May Jun Jul Aug Sep Nov Oct Nov Dec) } = (1 .. 12);
    return $months{$name}
}


while (<>) {
    if (my ($timestamp, $num, $type, $msg)
            = /^ <(.*)> ([0-9]+)\ (start|End) message ([0-9]+) $/x) {
        my ($day, $month, $year, $hour, $minute, $second) = split /[ :]/, $timestamp;
        my $dt = 'DateTime'->new(
                 year   => $year,
                 month  => month2num($month),
                 day    => $day,
                 hour   => $hour,
                 minute => $minute,
                 second => $second,
             );

        if ('start' eq $type) {
            die "Invalid start of $msg\n" if exists $message{$msg};
            $message{$msg} = { start => $dt, num => $num };

        } else {
            die "Invalid end of $msg\n" unless exists $message{$msg};
            my $duration = $dt->epoch - $message{$msg}{start}->epoch;
            say join ' | ', $message{$msg}{start},
                            "startmessage$msg",
                            "Endmessage$msg",
                            $message{$msg}{num},
                            "$duration seconds";
            delete $message{$msg};
        }
    }
}