For loop to produce a comma delimited string of appended values

Question

I have a script:

    OUTPUT_DIR=/share/es-ops/Build_Farm_Reports/WorkSpace_Reports
    BASE=/export/ws
    TODAY=`date +"%m-%d-%y"`
    HOSTNAME=`hostname`
    WORKSPACES=( "bob_avail" "bob_used" "mel_avail" "mel_used" "sideshow-ws2_avail" "sideshow-ws2_used" )
    if ! [ -f $OUTPUT_DIR/$HOSTNAME.csv ] && [ $HOSTNAME == "sideshow" ]; then
    echo "$HOSTNAME" >  $OUTPUT_DIR/$HOSTNAME.csv # with a linebreak
    separator="," # defined empty for the first value
    for v in "${WORKSPACES[@]}"
    do
      echo -n "$separator$v" >> $OUTPUT_DIR/$HOSTNAME.csv # append, concatenated, the separator and the value to the file
      #separator="," # comma for the next values
    done
    echo >> $OUTPUT_DIR/$HOSTNAME.csv # add a linebreak (if you want it)
    fi
    WORKSPACES2=( "bob" "mel" "sideshow-ws2" )
    separator="" # defined empty for the first value
    for v in "${WORKSPACES2[@]}"
    do
        echo -n  "$separator`df -m $BASE/$v | awk '{if (NR!=1) {print $3","$2}}'`" >> $OUTPUT_DIR/$HOSTNAME.csv
        separator="," # comma for the next values
    done

which produces this:

sideshow
,bob_avail,bob_used,mel_avail,mel_used,sideshow-ws2_avail,sideshow-ws2_used
470400,1032124,661826,1032124,43443,1032108

But I wanted the second for loop to put the date in the first column like this:

09-20-14,470400,1032124,661826,1032124,43443,1032108

meaning

$TODAY,,,mel avail>,,,

So the overall output would look like this:

sideshow
,bob_avail,bob_used,mel_avail,mel_used,sideshow-ws2_avail,sideshow-ws2_used
09-20-14,470400,1032124,661826,1032124,43443,1032108

and if this runs off a cron then I only have to check if the hostname is x and the .csv file does exist then:

sideshow
,bob_avail,bob_used,mel_avail,mel_used,sideshow-ws2_avail,sideshow-ws2_used
09-20-14,470400,1032124,661826,1032124,43443,1032108
09-20-15,470400,1032124,661826,1032124,43443,1032108

just add the next day's values.

rici · Accepted Answer

First off, there are a few simple ways to produce comma-separated lists. The one I usually use is paste -sd,, which requires the values to be on separate lines. That can easily be arranged:

printf %s\n "${WORKSPACES[@]}" | paste -sd,

but if we're going to pipe printf through a utility, we might as well put the commas in with printf and delete the one we don't want:

printf ,%s "{WORKSPACES[@]}" | cut -c2-

cut and paste will both guarantee that there is a newline at the end of their output. In this case, it seems to be what's desired, but if it were not, you can use "$(...)" to eliminate the newline:

printf %s "$(printf ,%s "{WORKSPACES[@]}" | cut -c2-)"

Extra added bonus: produce the _avail and _used labels with a bash one-liner:

paste -d, <(printf %s_avail\n "${WORKSPACES2[@]}") \
          <(printf %s_used\n "${WORKSPACES2[@]}") | paste -sd,

Now, let's consider the df invocation. You're invoking df six times in order to extract individual fields using a very powerful tool (awk) which could do pretty well all the work for you. Let's do that. First, we'll tell df about all the filesystems we want, and then we can handle all of the lines as we get them, extracting two fields and outputting them with commas in between. As an extra added bonus, we can output the timestamp, too:

WORKSPACES2=( "bob" "mel" "sideshow-ws2" )
df -m "${WORKSPACES2[@]/#//export/ws/}" | awk '
    BEGIN  { print strftime("%m-%d-%y")}
    NR > 1 { printf ",%s,%s", $3, $2; }
    END    { printf "
"}' >> "$OUTPUT_DIR/$HOSTNAME.csv"

The above assumes Gnu awk, which has a strftime function. With other awks, you have to invoke a shell:

df -m "${WORKSPACES2[@]/#//export/ws/}" | awk '
    BEGIN  { "date +%m-%d-%y" | getline date;
             printf "%s",date }
    NR > 1 { printf ",%s,%s", $3, $2; }
    END    { printf "
"}' >> "$OUTPUT_DIR/$HOSTNAME.csv"

The bash expression "${WORKSPACES2[@]/#//export/ws/}" is an iterated search and replace; that is, the search-and-replace is applied to each element of the array in turn, creating a separate "word" for each element. The pattern in this case is # which in here means "an empty pattern starting at the beginning of the line". The replacement starts immediately after the second /, and we don't (and must not) backslash-escape the following / because bash is not expecting a trailing / in the replacement (and the backslashes would be copied into the replacement if we put them in). I encourage you to experiment with echo and various search and replace strings in order to get the hang of it.

For loop to produce a comma delimited string of appended values

Answers (2)

Related Questions