CODEWITHSUNDEEP
bashshell
Matt
Matt

Reputation: 19

How to process previous 6 months log to the current months and future upcoming months without manually editing the bash script?

How to process previous 6 months log to the current months and future upcoming months without manually editing the bash script? My apologies as i am unable to post an image of it as i don't have 10 reputations yet.

This is the output i want:

Month     CBS      GFS      HR   HR    Payroll   INCV
          cbs1      gfs1    hr1  hr2    hrm      incv1
2013-07     97      89      14    28    30        4
2013-08     58     103      18    6     24        18
2013-09     54     110      11    14    25        17
2013-10     108    129      17    8     23        18
2013-11     52     137      12    8     21        30
2013-12     18     84       6     0     13        13
2014-01     8      6        1     0     9         3

Below is my bash script:

#!/bin/bash


for hostip in `cat $CONFIG `
do

  name=`echo $hostip|cut -d\: -f1`
  ip=`echo $hostip|cut -d\: -f2`
  HTML=/tmp/test.html
  rm -f $HTML
  touch $HTML

  echo "<link rel=\"stylesheet\" type=\"text/css\" href=\"/soc/stylesheet.css\" />" >> $HTML



   echo "<h2><b>Servers</b></h2>" >> $HTML
   echo "<table border="1">" >> $HTML
   echo "<tr><th rowspan="2">Month</th><th>CBS</th><th>GFS</th><th colspan="2">HR</th><th>Payroll</th><th>INCV</th></tr>" >> $HTML
   echo "<tr><th>cbs1</th><th>gfs1</th><th>hr1</th><th>hr1</th><th>hrm</th><th>incv1</th></tr>" >> $HTML

  How do i simplify running the different servers starting from 6 months 
  ago till now so that it will updating without me manually editing the script ?
  ###GREP The logs for the whole month without redundant data and do a word count

   MONTH=`date -d"6 month ago" +"%Y-%m"`
   LOG=/var/log/syslog-ng/svrs/sec2tes1
   CBS=`bzcat $LOG/2013-07*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
   GFS=`bzcat $LOG/2013-07*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
   HR1=`bzcat $LOG/2013-07*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
   HR2=`bzcat $LOG/2013-07*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
   PAYROLL=`bzcat $LOG/2013-07*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
   INCV=`bzcat $LOG/2013-07*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`

   MONTH1=`date -d"5 month ago" +"%Y-%m"`
   CBS_AUG=`bzcat $LOG/2013-08*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
   GFS_AUG=`bzcat $LOG/2013-08*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
   HR1_AUG=`bzcat $LOG/2013-08*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
   HR2_AUG=`bzcat $LOG/2013-08*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
   PAYROLL_AUG=`bzcat $LOG/2013-08*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
   INCV_AUG=`bzcat $LOG/2013-08*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`

   MONTH2=`date -d"4 month ago" +"%Y-%m"`
   CBS_SEP=`bzcat $LOG/2013-09*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
   GFS_SEP=`bzcat $LOG/2013-09*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
   HR1_SEP=`bzcat $LOG/2013-09*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
   HR2_SEP=`bzcat $LOG/2013-09*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
   PAYROLL_SEP=`bzcat $LOG/2013-09*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
   INCV_SEP=`bzcat $LOG/2013-09*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`

   MONTH3=`date -d"3 month ago" +"%Y-%m"`
   CBS_OCT=`bzcat $LOG/2013-10*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
   GFS_OCT=`bzcat $LOG/2013-10*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
   HR1_OCT=`bzcat $LOG/2013-10*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
   HR2_OCT=`bzcat $LOG/2013-10*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
   PAYROLL_OCT=`bzcat $LOG/2013-10*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
   INCV_OCT=`bzcat $LOG/2013-10*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`

   MONTH4=`date -d"2 month ago" +"%Y-%m"`
   CBS_NOV=`bzcat $LOG/2013-11*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
   GFS_NOV=`bzcat $LOG/2013-11*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
   HR1_NOV=`bzcat $LOG/2013-11*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
   HR2_NOV=`bzcat $LOG/2013-11*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
   PAYROLL_NOV=`bzcat $LOG/2013-11*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
   INCV_NOV=`bzcat $LOG/2013-11*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`

   MONTH5=`date -d"1 month ago"  +"%Y-%m"`
   CBS_DEC=`bzcat $LOG/2013-12*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
   GFS_DEC=`bzcat $LOG/2013-12*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
   HR1_DEC=`bzcat $LOG/2013-12*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
   HR2_DEC=`bzcat $LOG/2013-12*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
   PAYROLL_DEC=`bzcat $LOG/2013-12*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
   INCV_DEC=`bzcat $LOG/2013-12*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`

   MONTH6=`date +"%Y-%m"`
   CBS_JAN=`bzcat $LOG/2014-01*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
   GFS_JAN=`bzcat $LOG/2014-01*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
   HR1_JAN=`bzcat $LOG/2014-01*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
   HR2_JAN=`bzcat $LOG/2014-01*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
   PAYROLL_JAN=`bzcat $LOG/2014-01*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
   INCV_JAN=`bzcat $LOG/2014-01*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`




 ##Input the data from the logs into each cell of the HTML <td> tag

   echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.2013-07.html>$MONTH</a><td>$CBS<td>$GFS<td>$HR1<td>$HR2<td>$PAYROLL<td>$INCV</tr>" >>$HTML
   echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.2013-08.html>$MONTH1</a><td>$CBS_AUG<td>$GFS_AUG<td>$HR1_AUG<td>$HR2_AUG<td>$PAYROLL_AUG<td>$INCV_AU
G</tr>" >>$HTML
   echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.2013-09.html>$MONTH2</a><td>$CBS_SEP<td>$GFS_SEP<td>$HR1_SEP<td>$HR2_SEP<td>$PAYROLL_SEP<td>$INCV_SE
P</tr>" >>$HTML
   echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.2013-10.html>$MONTH3</a><td>$CBS_OCT<td>$GFS_OCT<td>$HR1_OCT<td>$HR2_OCT<td>$PAYROLL_OCT<td>$INCV_OC
T</tr>" >>$HTML
   echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.2013-11.html>$MONTH4</a><td>$CBS_NOV<td>$GFS_NOV<td>$HR1_NOV<td>$HR2_NOV<td>$PAYROLL_NOV<td>$INCV_NO
V</tr>" >>$HTML
   echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.2013-12.html>$MONTH5</a><td>$CBS_DEC<td>$GFS_DEC<td>$HR1_DEC<td>$HR2_DEC<td>$PAYROLL_DEC<td>$INCV_DE
C</tr>" >>$HTML
  echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.2014-01.html>$MONTH6</a><td>$CBS_JAN<td>$GFS_JAN<td>$HR1_JAN<td>$HR2_JAN<td>$PAYROLL_JAN<td>$INCV_JAN
</tr>" >>$HTML


   echo "</table>" >> $HTML

   echo "<br/>" >> $HTML
   echo "<br/>" >> $HTML

  cat - > /var/www/html/soc/test/archive/$HTML"


  rm -f $HTML


  done

Upvotes: 0

Views: 90

Answers (1)

Tim Pierce
Tim Pierce

Reputation: 5664

You seem to have the answer already: MONTH=`date -d"6 month ago" +"%Y-%m"` should give you the string 2013-07 (when it's run in January 2014, anyway).

Every place in your script where you literally write 2013-07 and so on, you can just use $MONTH instead. You can make the script more concise to boot by wrapping it in a small loop:

echo "<h2><b>Servers</b></h2>" >> $HTML
echo "<table border="1">" >> $HTML
echo "<tr><th rowspan="2">Month</th><th>CBS</th><th>GFS</th><th colspan="2">HR</th><th>Payroll</th><th>INCV</th></tr>" >> $HTML
echo "<tr><th>cbs1</th><th>gfs1</th><th>hr1</th><th>hr1</th><th>hrm</th><th>incv1</th></tr>" >> $HTML

for count in 6 5 4 3 2 1 0
do
    MONTH=`date -d"$count month ago" +"%Y-%m"`
    CBS=`bzcat $LOG/$MONTH*.log.bz2|grep 10.55.22.41 |sort|uniq | wc -l`
    GFS=`bzcat $LOG/$MONTH*.log.bz2|grep 10.55.22.31 |sort|uniq | wc -l`
    HR1=`bzcat $LOG/$MONTH*.log.bz2|grep 10.55.10.1 |sort|uniq | wc -l`
    HR2=`bzcat $LOG/$MONTH*.log.bz2|grep 10.55.21.12 |sort|uniq | wc -l`
    PAYROLL=`bzcat $LOG/$MONTH*.log.bz2|grep 10.55.21.18 |sort|uniq | wc -l`
    INCV=`bzcat $LOG/$MONTH*.log.bz2|grep 10.55.22.71 |sort|uniq | wc -l`
    echo " <tr><th><a href=/soc/test/archive/tmp/tripwire.$MONTH.html>$MONTH</a><td>$CBS<td>$GFS<td>$HR1<td>$HR2<td>$PAYROLL<td>$INCV</tr>" >>$HTML
done

echo "</table>" >> $HTML

Does that get you closer to a solution?

Upvotes: 1

Related Questions