compare lines with a pattern in two files

Question

lets take an example

If my data set is like this. LOG 1 (x.log) contains

INFO @1102266 PHResourceLayer_Z4: mti_clk_chk:################ start of test ################ ; T=1102266
INFO @1102334 PHResourceLayer_Z4: mti_clk_chk:Checking the period of MTI, MTI10 clk from SV; T=1102334

LOG 2 (y.log) contains

UVM_INFO @1092507 reporter Z4_COREA: mti_clk_chk: ################ start of test ################ ; T=1092507
UVM_INFO @1092563 reporter Z4_COREA: mti_clk_chk: Checking the period of MTI, MTI10 clk from SV; T=1092563

Then for first line I have to check

################ start of test ################ ; T=1102266

and

################ start of test ################ ; T=1092507

as value of T are not same so it should give these details in output file stating the details are not matching.

Similarily for line 2 we have to match

Checking the period of MTI, MTI10 clk from SV; T=1102334

and

Checking the period of MTI, MTI10 clk from SV; T=1092563

Here also values of T are not matching so pass it to output file.

I have to compare details line by line in two log files having certain keyword mti_clk_chk. Till now I'm able to parse it line by line with required keyword from both files into a third file. Now I want to compare the data present after the keyword following colon(:) and in output file I have to print the mismatched lines from first data-set on comparing it with second data-set and number of lines in second data set that are not present in first data-set. Data after parsing two log files is given below. Please help me out how to compare details provided in each line between two set of data.

open(FILE, "<x.log");
my @array = <FILE>;
close(FILE);

open(FILE, "<y.log");
my @array1 = <FILE>;
close(FILE);

open(FILE, ">>file.txt");
my @array2 = <FILE>;
foreach $_ (@array & @array1) {
         @array2 = grep {$_ =~ "mti_clk_chk:"} (@array);
                  print FILE "@array2";
          print FILE "\n \n \n";

         @array2 = grep {$_ =~ "mti_clk_chk:"} (@array1);
                  print FILE "@array2";
                  close(FILE);
exit;
}

Sample data in file.txt after parsing two input logs (x.log and y.log)

INFO @576892 mti_clk_chk: run_stimulus called; T=576892
INFO @1102266 PHResourceLayer_Z4: mti_clk_chk:################ start of test ################ ; T=1102266
INFO @1102334 PHResourceLayer_Z4: mti_clk_chk:Checking the period of MTI, MTI10 clk from SV; T=1102334
INFO @1102372 mti_clk_chk: Checking period of MTI CLk; T=1102372
INFO @1102377 mti_clk_chk: Period value of MTI Clock: 3.125000 ns; T=1102377
INFO @1102377 mti_clk_chk: MTI Clock is being generated correctly ; T=1102377
INFO @1102377 mti_clk_chk: Checking period of MTI10 CLk; T=1102377
INFO @1102418 mti_clk_chk: Period value of MTI10 Clock: 31.250000 ns; T=1102418
INFO @1102418 mti_clk_chk: MTI10 Clock is being generated correctly ; T=1102418
INFO @1102717 PHResourceLayer_Z4: mti_clk_chk: All clock period Checking done; T=1102717
INFO @1148661 mti_clk_chk: C-Code exit execution. code=<aa>; T=1148661
INFO @1148661 mti_clk_chk: ************************ SV END******************** ; T=1148661

UVM_INFO @0 reporter testbench.top_level_module.\mti_clk_chk::main : MTI_CLK_CHK_STIM Started .....; T=0
UVM_INFO @0 reporter testbench.top_level_module.\mti_clk_chk::main : run_stimulus called; T=0
UVM_INFO @1092507 reporter Z4_COREA: mti_clk_chk: ################ start of test ################ ; T=1092507
UVM_INFO @1092563 reporter Z4_COREA: mti_clk_chk: Checking the period of MTI, MTI10 clk from SV; T=1092563
UVM_INFO @1092598 reporter testbench.top_level_module.\mti_clk_chk::main : Checking period of MTI CLk; T=1092598
UVM_INFO @1092605 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipdss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(147) uvm_test_top.default_env.default_sequencer100@@mti_clk_chk mti_clk_chk:INFO: Period value of MTI Clock: 3.125000 ns; T=1092605
UVM_INFO @1092605 reporter testbench.top_level_module.\mti_clk_chk::main : MTI Clock is being generated correctly ; T=1092605
UVM_INFO @1092605 reporter testbench.top_level_module.\mti_clk_chk::main : Checking period of MTI10 CLk; T=1092605
UVM_INFO @1092655 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipdss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(165) uvm_test_top.default_env.default_sequencer100@@mti_clk_chk mti_clk_chk:INFO: Period value of MTI10 Clock: 31.250000 ns; T=1092655
UVM_INFO @1092655 reporter testbench.top_level_module.\mti_clk_chk::main : MTI10 Clock is being generated correctly ; T=1092655
UVM_INFO @1092850 reporter Z4_COREA: mti_clk_chk: All clock period Checking done; T=1092850
UVM_INFO @1092886 /proj/rru2_verif/usr/Tilak/SV_UVM/testbench/data_ipdss/v_ms_mti_stim_vip/testbench/classes_v/mti_clk_chk.sv(186) uvm_test_top.default_env.default_sequencer100@@mti_clk_chk mti_clk_chk:INFO: ************************ SV END******************** ; T=1092886

Answer 1

If I understood your input data correctly you want

• read lines from file 1
  • filter the lines that contain the filter keyword mti_clk_chk:
  • store everything after the keyword for comparison
• same with file 2
• print out lines from file 1 whose comparison string can't be found in file 2
• vice versa for file 2

The proposed solution for your question:

#!/usr/bin/perl
use warnings;
use strict;
use autodie;
use feature qw(say);

die "usage: $0 <log1> <log2>\n"
    if @ARGV < 2;
my($log1, $log2) = @ARGV;

# log file extractor function
sub extractor($) {
    my($file) = @_;
    my %lines;
    my @order;

    # Parse log file contents
    open(my $fh, '<', $file);
    while (<$fh>) {
        chomp;
        if (my($key) = /mti_clk_chk:\s*(.+)$/) {
            die "duplicate log line '$_' detected at ${file}:$.!\n"
                if exists $lines{$key};
            $lines{$key} = $_;
            push(@order, $key);
        }
    }
    close($fh);

    return((\%lines, \@order));
}

# parse log files
my($lines_log1, $order_log1) = extractor($log1);
my($lines_log2, $order_log2) = extractor($log2);

# lines in log1 but not in log2
say foreach (
    map  { $lines_log1->{$_} }
    grep { ! exists $lines_log2->{$_} }
    @{ $order_log1 }
);

# separator in output
say "";

# lines in log2 but not in log1
say foreach (
    map  { $lines_log2->{$_} }
    grep { ! exists $lines_log1->{$_} }
    @{ $order_log2 }
);

exit 0;

Test run with the two lines you have given as example. I added some junk at the start and the end to make sure it does not end up in the desired output.

$ cat dummy1.txt
test1
INFO @1102266 PHResourceLayer_Z4: mti_clk_chk:################ start of test ################ ; T=1102266
INFO @1102334 PHResourceLayer_Z4: mti_clk_chk:Checking the period of MTI, MTI10 clk from SV; T=1102334
test1

$ cat dummy2.txt
test2
UVM_INFO @1092507 reporter Z4_COREA: mti_clk_chk: ################ start of test ################ ; T=1092507
UVM_INFO @1092563 reporter Z4_COREA: mti_clk_chk: Checking the period of MTI, MTI10 clk from SV; T=1092563
test2

$ perl dummy.pl dummy1.txt dummy2.txt
INFO @1102266 PHResourceLayer_Z4: mti_clk_chk:################ start of test ################ ; T=1102266
INFO @1102334 PHResourceLayer_Z4: mti_clk_chk:Checking the period of MTI, MTI10 clk from SV; T=1102334

UVM_INFO @1092507 reporter Z4_COREA: mti_clk_chk: ################ start of test ################ ; T=1092507
UVM_INFO @1092563 reporter Z4_COREA: mti_clk_chk: Checking the period of MTI, MTI10 clk from SV; T=1092563