CODEWITHSUNDEEP
bashubuntuawksedgrep
user674669
user674669

Reputation: 12362

How do I filter out multiple lines which starts and ends with a given pattern?

In my company, we have a huge log file with java stacktraces. In general, its format is:

useful line 1
useful line 2
useful line 3
MARKER1 start of exception
... <--Around 100 lines here
end of exception MARKER2
useful line 4
useful line 5
useful line 6
MARKER1 start of exception
... <--Around 100 lines here
end of exception MARKER2
useful line 7

It has useful information mixed with useless exceptions.

Is it possible to filter out the entire contents of useless exceptions from the logs using a combination of awk/sed/grep..?

In the example above, the output would be:

useful line 1
useful line 2
useful line 3
useful line 4
useful line 5
useful line 6
useful line 7

Thanks.

Upvotes: 0

Views: 2611

Answers (4)

Alessandro Blasetti
Alessandro Blasetti

Reputation: 229

I think 

cat filename | grep useful

will work

Upvotes: 0

dawg
dawg

Reputation: 103884

Given your input, you can do:

$ awk 'BEGIN{ flag=1 } /MARKER/ {flag=!flag; next} flag' file
useful line 1
useful line 2
useful line 3
useful line 4
useful line 5
useful line 6
useful line 7

As pointed out in comments, you can also do:

awk '/MARKER/{f=!f;next} !f' file

Upvotes: 2

karakfa
karakfa

Reputation: 67507

another sed with anchored patterns

$ sed '/^MARKER1/,/MARKER2$/d' file

useful line 1
useful line 2
useful line 3
useful line 4
useful line 5
useful line 6
useful line 7

or translated to awk

$ awk '/^MARKER1/,/MARKER2$/{next} 1' file

Upvotes: 3

John1024
John1024

Reputation: 113864

Using awk

To exclude the start and end of exceptions and everything in between:

$ awk '/start of exception/,/end of exception/{next} 1' file
useful line 1
useful line 2
useful line 3
useful line 4
useful line 5
useful line 6
useful line 7

How it works:

  • /start of exception/,/end of exception/{next}

    For any line in the range from the start to the end of the exception, we skip the rest of the commands and start over on the next line.

  • 1

    For any other lines, we print them. 1 is awk's shorthand for print-the-line.

Using sed

$ sed '/start of exception/,/end of exception/d' file
useful line 1
useful line 2
useful line 3
useful line 4
useful line 5
useful line 6
useful line 7

How it works:

  • /start of exception/,/end of exception/d

    For any line in the range from the start to the end of the exception, we delete the line (d).

All other lines are, by default, printed.

Upvotes: 3

Related Questions