CODEWITHSUNDEEP
regexlinuxbashshellgrep
Srini V
Srini V

Reputation: 11375

parse Log File, check for date, report results

I need to take the time stamp printed in After FTP connection and check whether it happened today.

I have a log file which contains the following:

---------------------------------------------------------------------
Opening connection for file1.dat 
---------------------------------------------------------------------
---------------------------------------------------------------------
Before ftp connection -- time is -- Mon Oct 21 04:01:52 CEST 2013 
---------------------------------------------------------------------
---------------------------------------------------------------------
After  ftp connection -- time is Mon Oct 21 04:02:03 CEST 2013 .
---------------------------------------------------------------------
---------------------------------------------------------------------
Opening connection for file2.dat
---------------------------------------------------------------------
---------------------------------------------------------------------
Before ftp connection -- time is -- Wed Oct 23 04:02:03 CEST 2013 
---------------------------------------------------------------------
---------------------------------------------------------------------
After  ftp connection -- time is Wed Oct 23 04:02:04 CEST 2013 .
---------------------------------------------------------------------

Desired Output: 

INPUT:file1.dat --> FAIL # since it is Oct 21st considering today is Oct 23.
INPUT:file2.dat --> PASS # since it is Oct 23rd.
INPUT:file3.dat --> FAIL # File information does not exist

What I tried so far:

grep "file1.dat\\|Before ftp connection\\|After  ftp connection" logfilename

But this returns all the info that matches either file1.dat OR Before ftp connection OR After ftp connection. Considering the above sample, I get 5 lines out of which last 2 lines are from file2.dat:

Opening connection for file1.dat 
Before ftp connection -- time is -- Mon Oct 21 04:01:52 CEST 2013 
After  ftp connection -- time is Mon Oct 21 04:02:03 CEST 2013 .
Before ftp connection -- time is -- Wed Oct 23 04:02:03 CEST 2013 
After  ftp connection -- time is Wed Oct 23 01:02:04 CEST 2013 .

I am stuck here. So ideally I need to take Mon Oct 21 04:02:03 CEST 2013 and compare and print the a result FAIL.

Upvotes: 3

Views: 323

Answers (3)

anubhava
anubhava

Reputation: 786289

Use awk:

# read dates in shell variables
read x m d x x y < <(date)

awk -v f='file2.dat' -v m=$m -v d=$d -v y=$y '$0 ~ f {s=1; next} 
   s && /After  ftp connection/ {
       res = ($8==m && $9==d && $12==y) ? "PASS" : "FAIL";
       print f, res; exit
   }' file.log

file2.dat PASS

FOLLOW UP by OP:

I achieved the intended results by this:

check_success () 
{

    CHK_DIR=/Archive

    if [[ ! -d ${CHK_DIR} ]]; then 
        exit 1
    elif [[ ! -d ${LOG_FOLDER} ]]; then 
        exit 1
    fi

    count_of_files=$(ls -al --time-style=+%D $CHK_DIR/*.dat  | grep $(date +%D) | cut -f1 | awk '{ print $7}' | wc -l)

    if [[ $count_of_files -lt 1 ]]; then 
        exit 2
    fi

    list_of_files=$(basename $(ls -al --time-style=+%D $CHK_DIR/*.dat  | grep $(date +%D) | cut -f1 | awk '{ print $7}'))

    for filename in $list_of_files
    do
        filename=basename filename
        lg_name=$(grep -El "Opening.*$filename" $LOG_FOLDER/* | head -1 )
        m=$(date +%b)
        d=$(date +%d)
        y=$(date +%Y)
        output=$(awk -v f=$filename -v m=$m -v d=$d -v y=$y '$0 ~ f {s=1; next} s && /After  ftp connection/ { res = ($8==m && $9==d && $12==y) ? "0" : "1"; print res; exit }' $lg_name)

        if [[ ${output} != 0 ]]; then
            exit 2
        fi
    done
    exit 0
}

I used Anubhava's snippet, nevertheless Thanks to all the three champs.

Upvotes: 2

fedorqui
fedorqui

Reputation: 290515

It was tricky!

$ awk -vtoday=$(date "+%Y%m%d")
   '/^Opening/ {file=$4}
    /^After  ftp connection/ 
         {$1=$2=$3=$4=$5=$6=$NF="";
          r="date -d \"" $0 "\" \"+%Y%m%d\""; r | getline dat;
          if (today==dat) {print file, "PASS"} 
          else {print file, "FAIL"}}
 ' file
For file1.dat FAIL
For file2.dat PASS

Explanation

  • -vtoday=$(date "+%Y%m%d") gives today's date with "20131023" format
  • /^Opening/ {file=$4} gets lines starting with Opening and store the filename, that happens to be in the 4th field.
  • /^After ftp connection/ on lines starting with "After ftp connection...", do:
  • {$1=$2=$3=$4=$5=$6=$NF=""; delete up to 6th field and last one so the rest is the date info.
  • r="date -d \"" $0 "\" \"+%Y%m%d\""; r | getline dat; calculate the date on YYYYMMDD format of that line.
  • if (today==dat) {print file, "PASS} make comparison of dates.
  • else {print file, "FAIL"} idem.

Upvotes: 1

Chris Seymour
Chris Seymour

Reputation: 85883

Defining the records correctly makes things a lot easier:

$ awk '{print $5,($0~"After.*"d?"PASS":"FAIL")}' d="$(date +'%a %b %d')" RS= file
file1.dat FAIL
file2.dat PASS

Upvotes: 3

Related Questions