CODEWITHSUNDEEP
regexsed
Ken
Ken

Reputation: 4163

Using sed to delete all lines between two matching patterns

I have a file something like:

# ID 1
blah blah
blah blah
$ description 1
blah blah
# ID 2
blah
$ description 2
blah blah
blah blah

How can I use a sed command to delete all lines between the # and $ line? So the result will become:

# ID 1
$ description 1
blah blah
# ID 2
$ description 2
blah blah
blah blah

Can you please kindly give an explanation as well?

Upvotes: 115

Views: 101769

Answers (8)

Ed Morton
Ed Morton

Reputation: 203209

sed is great for doing s/old/new/ on individual lines but for anything else, just use awk for clarity, simplicity, portability, robustness, etc.

Using any awk in any shell on every Unix box:

$ awk '/\$/{f=0} !f{print} /#/{f=1}' file
# ID 1
$ description 1
blah blah
# ID 2
$ description 2
blah blah
blah blah

Upvotes: 1

potong
potong

Reputation: 58371

This might work for you (GNU sed):

sed '/^#/{:a;N;/^\$/M!ba;s/\n.*\n/\n/}' file

On encountering a line beginning # gather up further lines until one beginning $, then replace everything between newlines with a newline.

Otherwise print the result.

N.B. Ranges have a flaw in that they expect both a start and end regexp. If the second regexp is not forthcoming the action of the range is still invoked i.e. if the action of the range is to delete lines, this will continue deleting lines until the end of the file.

Upvotes: 1

javasuns
javasuns

Reputation: 1111

The example below removes lines between "if" and "end if".

All files are scanned, and lines between the two matching patterns are removed ( including them ).

IFS='
'
PATTERN_1="^if"
PATTERN_2="end if"

# Search for the 1st pattern in all files under the current directory.
GREP_RESULTS=(`grep -nRi "$PATTERN_1" .`)

# Go through each result
for line in "${GREP_RESULTS[@]}"; do

   # Save the file and line number where the match was found.
   FILE=${line%%:*}
   START_LINE=`echo "$line" | cut -f2 -d:`

   # Search on the same file for a match of the 2nd pattern. The search 
   # starts from the line where the 1st pattern was matched.
   GREP_RESULT=(`tail -n +${START_LINE} $FILE | grep -in "$PATTERN_2" | head -n1`)
   END_LINE="$(( $START_LINE + `echo "$GREP_RESULT" | cut -f1 -d:` - 1 ))"

   # Remove lines between first and second match from file
   sed -e "${START_LINE},${END_LINE}d;" $FILE > $FILE

done

Upvotes: -2

fbicknel
fbicknel

Reputation: 1363

In general form, if you have a file with contents of form abcde, where section a precedes pattern b, then section c precedes pattern d, then section e follows, and you apply the following sed commands, you get the following results.

In this demonstration, the output is represented by => abcde, where the letters show which sections would be in the output. Thus, ae shows an output of only sections a and e, ace would be sections a, c, and e, etc.

Note that if b or d appear in the output, those are the patterns appearing (i.e., they're treated as if they're sections in the output).

Also don't confuse the /d/ pattern with the command d. The command is always at the end in these demonstrations. The pattern is always between the //.

  • sed -n -e '/b/,/d/!p' abcde => ae
  • sed -n -e '/b/,/d/p' abcde => bcd
  • sed -n -e '/b/,/d/{//!p}' abcde => c
  • sed -n -e '/b/,/d/{//p}' abcde => bd
  • sed -e '/b/,/d/!d' abcde => bcd
  • sed -e '/b/,/d/d' abcde => ae
  • sed -e '/b/,/d/{//!d}' abcde => abde
  • sed -e '/b/,/d/{//d}' abcde => ace

Upvotes: 32

anubhava
anubhava

Reputation: 784948

Use this sed command to achieve that:

sed '/^#/,/^\$/{/^#/!{/^\$/!d}}' file.txt

Mac users (to prevent extra characters at the end of d command error) need to add semicolons before the closing brackets

sed '/^#/,/^\$/{/^#/!{/^\$/!d;};}' file.txt

OUTPUT

# ID 1
$ description 1
blah blah
# ID 2
$ description 2
blah blah
blah blah

Explanation:

  • /^#/,/^\$/ will match all the text between lines starting with # to lines starting with $. ^ is used for start of line character. $ is a special character so needs to be escaped.
  • /^#/! means do following if start of line is not #
  • /^$/! means do following if start of line is not $
  • d means delete

So overall it is first matching all the lines from ^# to ^\$ then from those matched lines finding lines that don't match ^# and don't match ^\$ and deleting them using d.

Upvotes: 116

SLePort
SLePort

Reputation: 15461

Another approach with sed:

sed '/^#/,/^\$/{//!d;};' file
  • /^#/,/^\$/: from line starting with # up to next line starting with $
  • //!d: delete all lines except those matching the address patterns

Upvotes: 27

Lri
Lri

Reputation: 27613

$ cat test
1
start
2
end
3
$ sed -n '1,/start/p;/end/,$p' test
1
start
end
3
$ sed '/start/,/end/d' test
1
3

Upvotes: 67

Ricardo Marimon
Ricardo Marimon

Reputation: 10687

I did something like this long time ago and it was something like:

sed -n -e "1,/# ID 1/ p" -e "/\$ description 1/,$ p"

Which is something like:

  • -n suppress all output
  • -e "1,/# ID 1/ p" execute from the first line until your pattern and p (print)
  • -e "/\$ description 1/,$ p" execute from the second pattern until the end and p (print).

I might be wrong with some of the escaping on the strings, so please double check.

Upvotes: 8

Related Questions