CODEWITHSUNDEEP
bash
user1987607
user1987607

Reputation: 2157

bash cat exclude multiple files based on grep results

I have the following cat command that I use in a bash script. I look for $SAMPLE.txt file in subfolders 20* and combine them into 1 output.txt

cat /$FOLDER/20*/$SAMPLE.txt > /$OUTPUTFOLDER/output.txt

I now want to exclude certain files conditionally.

I found the following here https://unix.stackexchange.com/questions/246048/cat-files-except-one

$ shopt -s extglob
$ cat -- !(DISCARD).txt > catKEPT

I want to do something like this.

Look for $SAMPLE and a pattern '$PAT1' in a $SAMPLEFILE. This $SAMPLEFILE is comma seperated. If there is a match, I want to store the first field of this line & use it to exclude files from cat

I would use this command to look for $SAMPLE and $PAT1 & then cut to keep my first field. I would assign that to a variable 'EXLUDE_FOLDER'

EXCLUDE_FOLDER=grep '$SAMPLE' $SAMPLEFILE | grep '$PAT1' | cut -d "," -f 1

And then use it like this

cat /$FOLDER/20*/$SAMPLE.txt -- !($FOLDER/$EXLUDE_FOLDER/$SAMPLE.txt) > /$OUTPUTFOLDER/output.txt

I'm stuck at putting this into an if/statement and dealing with situations where grep results in multiple matches, so multiple files should be excluded

Upvotes: 1

Views: 571

Answers (1)

Mark Setchell
Mark Setchell

Reputation: 207670

If SAMPLE and PAT are variables, you presumably want them expanded to their contents, which means you must put them in double quotes, not single quotes. Example:

SAMPLE=3

# Compare single quotes versus double
echo '$SAMPLE'              # outputs $SAMPLE
echo "$SAMPLE"              # outputs 3

If SAMPLEFILE is the name of a file, you must double-quote it, else it will fail if your filename has spaces in it, so you must use:

grep "$SAMPLE" "$SAMPLEFILE"

So, now you can test if your grep works like this:

grep "$SAMPLE" "$SAMPLEFILE" | grep "$PAT1" | cut -d "," -f 1

So, if that works, the next thing is that you want to capture the output of the command, so you need to use $(...). That means:

EXCLUDE_FOLDER=$(grep "$SAMPLE" "$SAMPLEFILE" | grep "$PAT1" | cut -d "," -f 1)

So, see test if that works now:

echo "$EXCLUDE_FOLDER"

Upvotes: 1

Related Questions