CODEWITHSUNDEEP
regexbashshellunixgrep
Mr D
Mr D

Reputation: 13

Grep logs between two timestamps in Shell

I am writing a script where I need to grep the logs exactly between two given timestamps . I don't want to use regex as it not full proof. Is there any other way through which I can achieve this ?

e.g: between time range 04:15:00 to 05:15:00

Log Format:

170.37.144.10 - - [17/Dec/2015:04:00:00 -0500] "GET /abc/def/ghi/xyz.jsp HTTP/1.1" 200 337 3440 0000FqZTmTG2yuMTJeny7hPDOvG
170.37.144.10 - - [17/Dec/2015:05:10:09 -0500] "POST /abc/def/ghi/xyz.jsp HTTP/1.1" 200 27 21124 0000FqZTmTG2yuMTJ

Upvotes: 1

Views: 1991

Answers (2)

Ed Morton
Ed Morton

Reputation: 203358

This might be what you want to do, using GNU awk for time functions:

$ cat tst.awk
BEGIN { FS="[][ ]+"; beg=t2s(beg); end=t2s(end) }
{ cur = t2s($4) }
(cur >= beg) && (cur <= end)

function t2s(time,      t) {
    split(time,t,/[\/:]/)
    t[2]=(match("JanFebMarAprMayJunJulAugSepOctNovDec",t[2])+2)/3
    return mktime(t[3]" "t[2]" "t[1]" "t[4]+0" "t[5]+0" "t[6]+0)
}

$ awk -v beg="17/Dec/2015:04:15" -v end="17/Dec/2015:05:15" -f tst.awk file
access_log.aging.20151217040207:170.37.144.10 - - [17/Dec/2015:05:10:09 -0500] "POST /abc/def/ghi/xyz.jsp HTTP/1.1" 200 27 21124 0000FqZTmTG2yuMTJ

but it's hard to guess without more sample input and expected output.

Upvotes: 2

Andrea Corbellini
Andrea Corbellini

Reputation: 17751

If you don't want to use regular expressions nor patterns for matching lines, then grep alone is not enough.

Here's a Bash+date solution:

# start and stop may be parameters of your script ("$1" and "$2"),
# here they are hardcoded for convenience.
start="17/Dec/2015 04:15:00 -0500"
stop="17/Dec/2015 05:15:00 -0500"

get_tstamp() {
    # '17/Dec/2015:05:10:09 -0500' -> '17/Dec/2015 05:10:09 -0500'
    datetime="${1/:/ }"
    # '17/Dec/2015 05:10:09 -0500' -> '17 Dec 2015 05:10:09 -0500'
    datetime="${datetime//// }"

    # datetime to unix timestamp
    date -d "$datetime" '+%s'
}

start=$(get_tstamp "$start")
stop=$(get_tstamp "$stop")

while read -r line
do
    datetime="${line%%:*}" # remove ':170.37.144.10 ...'
    tstamp="$(get_tstamp "$datetime")"

    # $tstamp now contains a number like 1450347009;
    # check if it is in range $start..$stop
    [[ "$tstamp" -ge "$start" && "$tstamp" -le "$stop" ]] && echo "$line"
done

Upvotes: 1

Related Questions