Reputation: 15430

How can I replace multiple empty lines with a single empty line in bash?

I have a file that contains:

something



something else

something else again

I need a bash command, sed/grep w.e that will produce the following output

something

something else

something else again

In other words, I need to replace multiple blank lines with just a single blank line. grep/sed are line based. I've never found a BASH solution that would work on multi-line regex patterns.

Upvotes: 63

Answers (14)

WKPlus

Reputation: 7255

I just solved this problem by sed. Even if this is a 7 year old question, someone may find this helpful, so I am writing my solution by sed here:

sed 'N;/^\n$/D;P;D;'

Upvotes: 40

Shaoyun

Reputation: 101

Use awk:

awk '{ /^\s*$/?b++:b=0; if (b<=1) print }' file

Breakdown:

/^\s*$/?b++:b=0
    - ? :       the ternary operator
    - /^\s*$/   matches a blank line
    - b         variable that counts consecutive blank lines (b++).
                however, if the current line is non-blank, b is reset to 0.


if (b<=1) print
    print if the current line is non-blank (b==0)
          or if there is only one blank line (b==1).

By adjusting the regex, you can generalize it to other scenarios like squeezing multiple blank lines (">") in email: https://stackoverflow.com/a/59189823/12483961

Upvotes: 5

marco

Reputation: 4665

For BSD-derived systems (including GNU):

You just need cat with the -s option which causes it to remove repeated empty lines from its output:

cat -s

From man page: -s --squeeze-blank: suppress repeated empty output lines.

Upvotes: 125

Robᵩ

Reputation: 168616

Python, with regular expression:

import re
import sys
sys.stdout.write(re.sub('\n{2,}','\n\n', sys.stdin.read()))

Upvotes: 2

biggusjimmus

Reputation: 2776

grep -A1 . <yourfile> | grep -v "^--$"

This grep solution works assuming you want the following:

Input

line1

line2
line3


line4



line5

Output

line1

line2
line3

line4

line5

Upvotes: 17

user879121

Reputation:

This uses marco's solution on multiple files:

for i in *; do FILE=$(cat -s "$i"); echo "$FILE" > "$i"; done

Upvotes: 3

clt60

Reputation: 63892

If someone want use perl

perl -00pe0 < file

will do the same, as cat -s :)

Upvotes: 4

mateusza

Reputation: 5743

Pipelining it to |uniq may be solution (if other than empty lines don't duplicate)

Upvotes: -6

Dan Moulding

Reputation: 220473

Usually, if I find that sed can't do something I need, I turn to awk:

awk '
BEGIN {
    blank = 0;
}

/^[[:blank:]]*$/ {
     if (!blank) {
          print;
     }
     blank = 1;
     next;
}

{
     print;
     blank = 0;
}' file

Upvotes: 4

mouviciel

Reputation: 67831

A solution with awk, which replaces several blank lines with a single blank line:

awk 'BEGIN{bl=0}/^$/{bl++;if(bl==1)print;else next}/^..*$/{bl=0;print}' myfile

Upvotes: 4

Clue Less

Reputation: 3165

I take it that you'll probably want to remove lines that only have whitespace.

That can be done with:

sed /^[:space:]*$/d FILE

Upvotes: -3

Can Berk Güder

Reputation: 113300

Actually, if you replace multiple newlines with a single newline, the output would be:

something
something else
something else again

You can achieve this by:

sed /^$/d FILE

Upvotes: 13

Carl Sverre

Reputation: 1188

Super easy to do with vim. Just open the file and type the following:

:%s/\n\n\n*/\r\r/

That will reduce all blocks of more than 2 new lines to 2 new lines. Hope this helps!

Upvotes: 0

Jonas Kölker

Reputation: 7837

Use python:

s = file("filename.txt").read()
while "\n\n\n" in s: s = s.replace("\n\n\n", "\n\n")
import sys
sys.stdout.write(s)

Upvotes: 2

How can I replace multiple empty lines with a single empty line in bash?

Answers (14)

Related Questions