CODEWITHSUNDEEP
bashglob
steiny
steiny

Reputation: 203

Expand shell glob in variable into array

In a bash script I have a variable containing a shell glob expression that I want to expand into an array of matching file names (nullglob turned on), like in

pat='dir/*.config'
files=($pat)

This works nicely, even for multiple patterns in $pat (e.g., pat="dir/*.config dir/*.conf), however, I cannot use escape characters in the pattern. Ideally, I would like to able to do

pat='"dir/*" dir/*.config "dir/file with spaces"'

to include the file *, all files ending in .config and file with spaces.

Is there an easy way to do this? (Without eval if possible.)

As the pattern is read from a file, I cannot place it in the array expression directly, as proposed in this answer (and various other places).

Edit:

To put things into context: What I am trying to do is to read a template file line-wise and process all lines like #include pattern. The includes are then resolved using the shell glob. As this tool is meant to be universal, I want to be able to include files with spaces and weird characters (like *).

The "main" loop reads like this:

    template_include_pat='^#include (.*)$'
    while IFS='' read -r line || [[ -n "$line" ]]; do
        if printf '%s' "$line" | grep -qE "$template_include_pat"; then
            glob=$(printf '%s' "$line" | sed -nrE "s/$template_include_pat/\\1/p")
            cwd=$(pwd -P)
            cd "$targetdir"
            files=($glob)
            for f in "${files[@]}"; do
                printf "\n\n%s\n" "# FILE $f" >> "$tempfile"
                cat "$f" >> "$tempfile" ||
                    die "Cannot read '$f'."
            done
            cd "$cwd"
        else
            echo "$line" >> "$tempfile"
        fi
    done < "$template"

Upvotes: 7

Views: 3300

Answers (1)

Charles Duffy
Charles Duffy

Reputation: 295726

Using the Python glob module:

#!/usr/bin/env bash

# Takes literal glob expressions on as argv; emits NUL-delimited match list on output
expand_globs() {
  python -c '
import sys, glob
for arg in sys.argv[1:]:
  for result in glob.iglob(arg):
    sys.stdout.write("%s\0" % (result,))
' _ "$@"
}

template_include_pat='^#include (.*)$'
template=${1:-/dev/stdin}

# record the patterns we were looking for
patterns=( )

while read -r line; do
  if [[ $line =~ $template_include_pat ]]; then
    patterns+=( "${BASH_REMATCH[1]}" )
  fi
done <"$template"

results=( )
while IFS= read -r -d '' name; do
  results+=( "$name" )
done < <(expand_globs "${patterns[@]}")

# Let's display our results:
{
  printf 'Searched for the following patterns, from template %q:\n' "$template"
  (( ${#patterns[@]} )) && printf ' - %q\n' "${patterns[@]}"
  echo
  echo "Found the following files:"
  (( ${#results[@]} )) && printf ' - %q\n' "${results[@]}"
} >&2

Upvotes: 1

Related Questions