CODEWITHSUNDEEP
bashsedawk
Kasper Jan Mooijman
Kasper Jan Mooijman

Reputation: 186

remove block of text between two lines based on content

I need to remove/filter a very large log file i managed to bring the log-file into blocks of text starting with a line containing <-- or --> ending with a line containing Content-Length: now if this block of text contains the word REGISTER it need to be deleted.

i found the flowing example:

 # sed script to delete a block if /regex/ matches inside it
 :t
 /start/,/end/ {    # For each line between these block markers..
    /end/!{         #   If we are not at the /end/ marker
       $!{          #     nor the last line of the file,
          N;        #     add the Next line to the pattern space
          bt
       }            #   and branch (loop back) to the :t label.
    }               # This line matches the /end/ marker.
    /regex/d;       # If /regex/ matches, delete the block.
 }                  # Otherwise, the block will be printed.
 #---end of script---

written by Russell Davies on this page

but i do not know how to transport this to a single line statement to use in a pipe my goal is to pipe a tail -F of the log file to the final version so it get updates by the minute

Upvotes: 1

Views: 4338

Answers (3)

potong
potong

Reputation: 58430

This might work for you (GNU sed);

sed '/<--\|-->/!b;:a;/Content-Length/!{$!{N;ba}};//{/REGISTER/d}' file
  • /<--\|-->/!b if a line does not contain <-- or --> print it
  • :a;/Content-Length/!{$!{N;ba}} keep appending lines until the string Content-Length or the end of file is encountered.
  • //{/REGISTER/d} if the line(s) read in contains Content-Length and REGISTER delete it/them else print it/them as normal.

Upvotes: 2

Ed Morton
Ed Morton

Reputation: 203645

Try this:

awk '/<--|-->/{rec=""; f=1} f{rec = rec $0 ORS} /Content-Length:/{ if (f && (rec !~ "REGISTER")) printf "%s",rec; f=0}' file

If it doesn't do what you want, provide more info on what you want along with sample input and output.

To break down the above, here's each statement on separate lines with some comments:

awk '
   /<--|-->/ {rec=""; f=1} # find the start of the record, reset the string to hold it and set a flag to indicate we've started processing a record
   f {rec = rec $0 ORS}    # append to the end of the string containing the current record
   /Content-Length:/{      # find the end of the record
      if (f && (rec !~ "REGISTER")) # print the record if it doesn't contain "REGISTER"
         printf "%s",rec
      f=0                  # clear the "found record" indicator
   }
' file

and if you have text between your records that you'd want printed, just add a test for the "found" flag not being set and invoke the default action of printing the current record (!f;)

awk '/<--|-->/{rec=""; f=1} f{rec = rec $0 ORS} !f; /Content-Length:/{ if (f && (rec !~ "REGISTER")) printf "%s",rec; f=0}' file

Upvotes: 3

Zsolt Botykai
Zsolt Botykai

Reputation: 51613

If I get what you need correctly, you want to filter out the block, that is this only print the block:

tail -f logfile | sed -n '/\(<--\|-->\)/,/Content-Length:/ p'

If you want to delete it:

tail -f logfile | sed '/\(<--\|-->\)/,/Content-Length:/ d'

Upvotes: 1

Related Questions