CODEWITHSUNDEEP
regexarraysbashvariablesawk
user1408456
user1408456

Reputation:

Passing a Bash Array Element to an Awk Regex Expression

I've found several questions on how to pass variables from bash into awk, most notably the -v command, but I can't quite seem to get them to do what I want.

Outside the script, the command I'm running is

awk '$2 ~ /^\/var$/ { print $1 }' /etc/fstab

Which searches /etc/fstab for JUST the /var partition, and should either print out the physical mount point, or if there isn't one, nothing at all.

Now inside the script I have an array that contains numerous partitions, and what I want to do is iterate through that array to search fstab for each physical mount point. The problem comes in at the fact that the elements in the array have a / in them.

So what I want to do (In horrifically incorrect awk) is:

PARTITIONS=(/usr /home /var tmp);
for ((n=0; n<${#PARTITION[@]}; n++)); do
    cat /etc/fstab | awk '$2 ~ /^\${PARTITIONS[$n]}$/ { print $1 }';
done

But I know that that's not correct. The closest I have right now is:

PARTITIONS=(/usr /home /var tmp);
for ((n=0; n<${#PARTITION[@]}; n++)); do
    cat /etc/fstab | awk -v partition="${PARTITIONS[$n]}" '$2 ~ /^\/var$/ { print $1," ",partition }';
done

Which at LEAST gets the partition variable into awk, but doesn't help me at all with matching it.

So basically, I need to feed the array in, and get the physical partitions back out. Eventually the results will be assigned to another array, but once I get the output I can go from there.

I also understand awk can remove the need for the cat at the beginning, but I don't know enough about awk to do that yet. :)

Thanks for any help.

EDIT

cat /etc/fstab | awk -v partition="${PARTITIONS[$n]}" '$2 ~ partition { print $1 }'

Approximates what I needed enough to be useful. I was focusing far too much on including the regex apparently. If anyone else could clean this up, it would be much appreciated :)

Upvotes: 3

Views: 2826

Answers (6)

Zsolt Botykai
Zsolt Botykai

Reputation: 51653

You can do it like this:

awk -v partitions="${PARTITIONS[*]}" '
    BEGIN { split(partitions,a," ") }
    { for (e in a) { if ($2 ~ a[e]) { print $1 } } }' /etc/fstab

So you don't need to create a for cycle outside awk and this means fewer processes.

Upvotes: 0

Thor
Thor

Reputation: 47189

You could also pass the whole array into awk through -v, this assumes that the directory names do not contain spaces:

PARTITIONS=(/usr /home /var /tmp)
awk -v partition="${PARTITIONS[*]}" \
  '$2 != "" && partition ~ $2"\\>" { print $1 }' /etc/fstab

This avoids the need for a for loop.

Explanation

  • `partition="${PARTITIONS[*]}" passes in the whole array as space separated string.
  • $2 != "" means no empty lines are matched.
  • partition ~ $2"\\>" matches $2 to the passed in string, \\> requires the match to be at the end of a word.

Upvotes: 2

geirha
geirha

Reputation: 6171

You can pass the array on as an additional file to awk, using bash's process substitution.

partitions=( /usr /home /var /tmp )
awk '
    FNR==NR { partitions[$0]=""; next } 
    $1 !~ /^#/ && ($2 in partitions) { print $1 }
' <(printf '%s\n' "${partitions[@]}") /etc/fstab

NR holds the current number of records (lines) read, and FNR holds the current number of records read in the current file, so FNR==NR is only true when reading the first file, which is the process substitution in this case. So you fill up the partitions array for the first file.

Then, for the second file, you just check if the second field is in the array...

In this case though, I'd just use bash (version >= 4.0), since /etc/fstab is typically fairly small.

declare -A 'partitions=([/usr]= [/home]= [/var]= [/tmp]=)'
while read -r spec file vfstype mntops freq passno; do
    [[ $spec != \#* && ${partitions[$file]+set} ]] && echo "$spec"
done < /etc/fstab

Or depending on the actual goal, you could parse df, which will tell you what filesystem the directory is on.

dirs=( /usr /home /var /tmp )
for dir in "${dirs[@]}"; do
    { read -r; read -r part _; } < <(df -P "$dir")
    echo "$part"
done

Upvotes: 1

Dennis Williamson
Dennis Williamson

Reputation: 360395

awk -v partition="${partitions[$n]}" '$2 ~ "^/" partition "$" { print $1 }' /etc/fstab

You can concatenate your regex characters (^ - beginning of string and $ - end of string) and the slash which is part of the partition name and the variable containing the partition name by placing them adjacent to each other. You don't need to use the slashes that delimit hard-coded regexes.

AWK will accept the filename as an argument without using cat to pipe it or using < to redirect it.

I recommend using mixed or lowercase variable names in the shell as a habit to avoid potential name collisions with shell or environment variables.

Upvotes: 2

DigitalRoss
DigitalRoss

Reputation: 146141

Here are some observations that may help.

If the input is just a single file, it isn't necessary to cat it to anything. That is:

$ cat file | program # would normally just be ...
$ program < file

If you need to feed something complicated to awk(1), then maybe you do have a use case for cat x | y ... you could do something like ...

(echo StartFlag ${PARTITIONS[*]}; cat /etc/fstab) | awk ...

And finally, for the best results on SO ask in a format like ...

  1. My PARTITIONS bash variable contains simplified example contents
  2. Suppose my /etc/fstab contains simplified example fstab
  3. How do I get the following output: exact desired output based on simplified input
  4. This is what I've tried: some people won't just provide code, it helps to ask for assistance on a specific programming problem where you have tried to reach a solution

Upvotes: 0

lynxlynxlynx
lynxlynxlynx

Reputation: 1433

First, to get the most annoying thing out of the way (GUoC), awk can work on a file just like cat, so just pass it directly. You can't pass whole arrays via -v unflattened, but since you're iterating over the items, it doesn't matter. If you want to avoid -v, you can pass bash variables by directly including them into awk scripts, you just have to be careful about the quoting (whitespace and awk's own $variable usage). Examples:

awk '$2 ~ "'${PARTITIONS[$n]}'" { print $1 }' /etc/fstab

Or the more complicated version with soft quotes:

awk "\$2 ~ /${PARTITIONS[$n]//\//\\/}/ { print \$1 }" /etc/fstab

Upvotes: 1

Related Questions