CODEWITHSUNDEEP
bash
Mateusz Chrzaszcz
Mateusz Chrzaszcz

Reputation: 1280

Excluding properties from bash script

Let's assume I have a few folders in such hierarchy:

/somewhere/firstDirectory
/somewhere/secondDirectory
/somewhere/DifferentOne
/somewhere/ImportantLogs
/somewhere/CFPC

(Plenty of different folders here as well)

I want to write a bash script, that will parse all files in a certain number of directories in these hierarchy, that might vary due to differences in environments it will be deployed on. Number of directories, their names and content of files that script should check might and will be different.

So, instead of creating one, universal script with plenty of logic branching I came to conlcusion it will be better to put what varies into properties file, like these exemplar two:

env1.properties:
 $LOG_DIR="/somewhere/[firstDirectory,CFPC]"
 $MATCHING_PATTERN="ERROR.*"

env2.properties:
  $LOG_DIR="/somewhere/[ImportantLogs,DifferentOne,CFPC]"
  $MATCHING_PATTERN="ERROR.*((Parsing ERROR with reason)$"

Then we have a script like this:

#!/bin/bash

. $CONFIG/common/error_mailer.properties

ENTRY_MAX_AGE=$(($MAX_AGE_IN_HOURS*60*60))
LOG_FILES=$(find $LOG_DIR -type f -name $FILE_PATTERN)
LINES_WITH_ERRORS=$(zegrep -h "$MATCHING_PATTERN" $LOG_FILES | awk -v max_age=$ENTRY_MAX_AGE 'BEGIN {FS="[:. ]"} {now=systime()} {then=mktime(20$3 " " $2 " " $1 " " $4 " " $5 " " $6); if((now-max_age)<=then) print $0}')

if [ ! -z "$LINES_WITH_ERRORS" ]
then
  COUNT=$(echo "$LINES_WITH_ERRORS" | wc -l )
  RESULT_TABLE=$(
    echo -e "$LINES_WITH_ERRORS" |
    sort |
    head -n $MAX_NUMBER_OF_ITEMS |
    gawk -vpattern="$MATCHING_PATTERN" 'BEGIN{printf "Date and Time\t\tError Item\\n"}  match($0, pattern, m){printf  $1 " " $2 "\t" m[1] ".\\n"}'
  )
  MESSAGE+="$COUNT\n\n"
  if [ $COUNT -ge $MAX_NUMBER_OF_ITEMS ]
  then
    MESSAGE+=$MAIL_SUMMARY_TRIM
  else
    MESSAGE+=$MAIL_SUMMARY
  fi
  MESSAGE+=$RESULT_TABLE
  MESSAGE+=$FOOTER
  SUBJECT="$TITLE - $TOTAL_NUMBER_MESSAGE$COUNT"
  MESSAGE=$(echo -e "$MESSAGE" | sed 's/$/\r/')
  echo -e "Sending message with subject '$SUBJECT' and body:\n$MESSAGE\n"
  echo -e "$MESSAGE" | mail -v -s "$SUBJECT" "$RECIPIENT_LIST"
else
  echo "Nothing to send"
fi

(TITLE, FOOTER, $TOTAL_NUMBER_MESSAGE, MAIL_SUMMARY_TRIM are unimportant for explaining this issue variables)

And I would like to ask, if what I try to do is correct? Or should it be done differently in bash? Especially that part: $LOG_DIR="/somewhere/[firstDirectory,CFPC]" is kinda fishy for me, as I don't know how to elegantly pass the folders I want to check. I couldn't find better way of solving this out. If you could advice me how to improve that script quality I will be grateful.

Upvotes: 0

Views: 63

Answers (1)

tripleee
tripleee

Reputation: 189327

Your wildcard is wrong. Where you are using square brackets, you should be using curly ones.

However, because Bash will not perform brace expansion in this context, I would change the variable into an array, and then change the calling code accordingly.

Additionally, the variables you are assigning to should not have a dollar sign. (The dollar sign is for interpolation, i.e. accessing the variable's value.)

LOG_DIRS=( /somewhere/{firstDirectory,CFPC} )
MATCHING_PATTERN="ERROR.*"

Then in your main script, change $LOG_DIR to "${LOG_DIRS[@]}" (notice that I renamed the variable to better reflect its contents).

In a regular expression used by grep and its variants, a trailing wildcard is useless; the program looks for the pattern anywhere in each input line.

Upvotes: 1

Related Questions