CODEWITHSUNDEEP
bashdateubuntulogging
Niklas
Niklas

Reputation: 465

Is there a way to delete all log entries in a file older than a certain date? (BASH)

I have log file in which I'm trying to delete all entries older than a specified date. Though I haven't succeeded with this yet. What I've tested so far is having an input for what the entries must be older than to be deleted and then loop like this:

#!/bin/bash

COUNTER=7
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -@${DATE} "+%Y-%m-%d")

while [ -n "$(grep $DATE test.txt)" ]; do

    sed -i "/$DATE/d" test.txt

    COUNTER=$((${COUNTER}+1))
    DATE=$(date -d "-${COUNTER} days" +%s)
    DATE=$(date -d @${DATE} +"%Y-%m-%d")
done

This kind of works except when a log entry doesn't exist for date. When it doesn't find a match it aborts the loop and the even older entries are kept.

Update

This was how I solved it:

#!/bin/bash

COUNTER=$((7+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -@${DATE} "+%Y-%m-%d")

if [ -z "$(grep $DATE test.txt)" ]; then
   exit 1
fi

sed -i "1,/$DATE/d" test.txt

Upvotes: 3

Views: 8212

Answers (3)

Abdullah Teke
Abdullah Teke

Reputation: 19

I deleted log records in syslog-ng files before 60 days ago with following code.

#!/bin/bash   

LOGFILE=/var/log/syslog    
DATE=`date +"%b %e" --date="-60days"`    
sed -i "/$DATE/d" $LOGFILE

Upvotes: 1

Niklas
Niklas

Reputation: 465

Sorry for answering my own question but I went with Martin Frost's suggestion in the comments. It was much easier than the other suggestions.

This was my implementation:

#!/bin/bash

# requirements for script script

COUNTER=$((7+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -@${DATE} "+%Y-%m-%d")

sed -i "1,/$DATE/d" test.txt

Thanks for all the help!

Upvotes: 2

Zsolt Botykai
Zsolt Botykai

Reputation: 51593

Depending on your logfile format, assuming that the timestamp is the first column in the file you can do it like this with (g)awk.

awk 'BEGIN { OneWeekEarlier=strftime("%Y-%m-%d",systime()-7*24*60*60) }
     $1 <= OneWeekEarlier { next }
     1' INPTUTLOG > OUTPUTLOG

This computes the date - surprise, surprise - one week earlier, then checks if the first column (white space separated columns by default) is less than or equal, and if true, skips the line, otherwise prints.

The hard part is doing the "in place" editing with awk. But it can be done:

{ rm LOGFILE && awk 'BEGIN { OneWeekEarlier=strftime("%Y-%m-%d",systime()-7*24*60*60) }
     $1 <= OneWeekEarlier { next }
     1' > LOGFILE ; } < LOGFILE

HTH

Upvotes: 1

Related Questions