CODEWITHSUNDEEP
linuxsedawk
Richard
Richard

Reputation: 1052

Remove a line surrounded by blank lines

Basically, I have output from nmap, which gives me an IP and then a list of open ports underneath that, followed by a blank line. I have filtered out the ports that I don't want anymore (grep -v http, for example), but I can't figure out how to remove the IP addesses that have no following ports.

Is there a way to do this with sed?

Sample data:

Nmap scan report for 1.1.1.1
3389/tcp  open  ms-term-serv
5357/tcp  open  unknown
5432/tcp  open  postgresql
8080/tcp  open  http-proxy
49152/tcp open  unknown
49153/tcp open  unknown
49154/tcp open  unknown
49155/tcp open  unknown

Nmap scan report for 2.2.2.2

Nmap scan report for 3.3.3.3
80/tcp    open  http
443/tcp   open  https
6646/tcp  open  unknown
8000/tcp  open  http-alt
49152/tcp open  unknown
49153/tcp open  unknown
49154/tcp open  unknown
49156/tcp open  unknown
49157/tcp open  unknown

Upvotes: 0

Views: 347

Answers (4)

user2070305
user2070305

Reputation: 487

I am infrequent awk user, but came across this question when looking for a way to delete a solitary lines of text surrounded by blank lines. The answer provided by @iiSeymour above is so elegant that I had to figure out how it worked and wanted to share my learning here.

Awk:

  • reads Records from the input separated by Record Separator (RS - default: newline)
  • uses the Field Separator to break a Record into Fields (FS - default: white space)
  • (conditionally) take an action. The default action is to print the record followed by the Output Record Separator (ORS) (default: newline)

Looking at this in detail

awk '$2' ORS='\n\n' FS='\n' RS=

When RS is set to the null string, records are separated by blank lines (subtle point on the man page). By my observation, the newlines until the blank line are retained, but the newline on the last line (preceding the blank line) is not included in the Record (more below). Note also that one or more blank lines separate the records, so that's how multiple consecutive blank lines in the input are absorbed.

We now have a record consisting of one or more lines of non-blank text. By setting FS to a newline, the Record is split into Fields of one input line each.

The awk program consists of an optional pattern followed by an action.

pattern {action}

Typically, we see the pattern is to match a regular express like /abc/ but note that, when used as a pattern, awk interprets an empty string as false and a non-empty string as true. (This was not readily found on the man page.)

When the Record contains (at least) 2 consecutive non-blank lines

  • (non-empty) $2 in the pattern position evaluates to true
  • there is no {action} so the default action is taken which is to print the record followed by the ORS which has been set to two newlines. The first newline replaces the dropped newline from when the record was read with a null RS (as noted above) and the second one to provide a single blank line between the blocks of text.

When there is one non-blank line followed by a blank line

  • (empty) $2 evaluates to false, no action is taken and the solitary line is dropped

Upvotes: 1

Chris Seymour
Chris Seymour

Reputation: 85883

With awk you can simply do awk '$2' ORS='\n\n' FS='\n' RS= file: 

$ awk '$2' ORS='\n\n' FS='\n' RS= file
Nmap scan report for 1.1.1.1
3389/tcp  open  ms-term-serv
5357/tcp  open  unknown
5432/tcp  open  postgresql
8080/tcp  open  http-proxy
49152/tcp open  unknown
49153/tcp open  unknown
49154/tcp open  unknown
49155/tcp open  unknown

Nmap scan report for 3.3.3.3
80/tcp    open  http
443/tcp   open  https
6646/tcp  open  unknown
8000/tcp  open  http-alt
49152/tcp open  unknown
49153/tcp open  unknown
49154/tcp open  unknown
49156/tcp open  unknown
49157/tcp open  unknown

If the extra newline added to the end of file in previous script is a problem then use this alternative: 

awk '/^Nmap/{h=$0;i=NR;next}NR==i+1{if($0){print h;p=1}else p=0}p' file

Upvotes: 2

Beta
Beta

Reputation: 99172

It's pretty easy in sed:

sed '$!N;/\n$/d' filename

Or if you want to alter the file in place:

sed -i '' '$!N;/\n$/d' filename

Upvotes: 0

MelBurslan
MelBurslan

Reputation: 2511

the following awk code segment worked for me for the fictitious file I created as such:

22.22.22.22
dflkhhdfjhdk
tslkdkffhdskjgh

33.33.33.33
tddfgkghdfkj

44.44.44.44

55.55.55.55
ghdkjghdkfjhjdfhg
iuryweiu
kjwhkjfh

66.66.66.66

77.77.77.77
cxlvbclbnc

and run this thru the following awk program:

awk '/[0-9][0-9].[0-9][0-9].[0-9][0-9].[0-9][0-9]/ { K=$0; next }
K { if(length) print K "\n" $0; K=""; next }
1
END { if(K) print K }' infile > outfile

the contents of outfile afterwards are as follows:

22.22.22.22
dflkhhdfjhdk
tslkdkffhdskjgh

33.33.33.33
tddfgkghdfkj

55.55.55.55
ghdkjghdkfjhjdfhg
iuryweiu
kjwhkjfh

77.77.77.77
cxlvbclbnc

The regex I used to match the IP address needs to be modified to match your particular case of course but I think you've already covered it.

Hope this helps. I know it is not sed but close enough I suppose.

Upvotes: 1

Related Questions