CODEWITHSUNDEEP
bashredhat
Joshua
Joshua

Reputation: 4320

Bash script to remove lines containing any of a list of words

I have a large config file that I use to define variables for a script to pull from it, each defined on a single line. It looks something like this:

var   val
foo   bar
foo1  bar1
foo2  bar2

I have gathered a list of out of date variables that I want to remove from the list. I could go through it manually, but I would like to do it with a script, which would be at least more stimulating. The file that contains the vlaues may contain multiple instances. The idea is to find the value, and if it's found, remove the entire line.

Does anyone know if this is possible? I know sed does this but I do not know how to make it use a file input.

Upvotes: 0

Views: 1725

Answers (2)

Flmhdfj
Flmhdfj

Reputation: 918

Here's one with sed. Add words to the array. Then use 

./script target_filename

(assuming you put the following in a file called script). (Not very efficient). I think it might be more efficient if we concat the words and put it in the regex like bbonev did

#!/bin/bash

declare -a array=("foo1" "foo2")

for i in "${array[@]}";
do
  sed -i "/^${i}\s.*/d" $1
done

It's actually even simpler using file input

If you have a word file

word1
word2
word3
.....

then the following will do the job

#!/bin/bash

while read i; 
do
  sed -i "/^${i}\s.*/d" $2
done <$1

usage:

./script wordlist target_file

Upvotes: 0

konsolebox
konsolebox

Reputation: 75488

#!/bin/bash
shopt -s extglob
REMOVE=(foo1 foo2)
IFS='|' eval 'PATTERN="@(${REMOVE[*]})"'
while read -r LINE; do
    read A B <<< "$LINE"
    [[ $A != $PATTERN ]] && echo "$LINE"
done < input_file.txt > output_file.txt

Or (Use with a copy first)

#!/bin/bash
shopt -s extglob
FILE=$1 REMOVE=("${@:2}")
IFS='|' eval 'PATTERN="@(${REMOVE[*]})"'
SAVE=()
while read -r LINE; do
    read A B <<< "$LINE"
    [[ $A != $PATTERN ]] && SAVE+=("$LINE")
done < "$FILE"
printf '%s\n' "${SAVE[@]}" > "$FILE"

Running with

bash script.sh your_config_file pattern1 pattern2 ...

Or

#!/bin/bash
shopt -s extglob
FILE=$1 PATTERNS_FILE=$2
readarray -t REMOVE < "$PATTERNS_FILE"
IFS='|' eval 'PATTERN="@(${REMOVE[*]})"'
SAVE=()
while read -r LINE; do
    read A B <<< "$LINE"
    [[ $A != $PATTERN ]] && SAVE+=("$LINE")
done < "$FILE"
printf '%s\n' "${SAVE[@]}" > "$FILE"

Running with

bash script.sh your_config_file patterns_file

Upvotes: 1

Related Questions