Reputation: 4145

How to use sed to remove only double empty lines?

I found this question and answer on how to remove triple empty lines. However, I need the same only for double empty lines. Ie. all double blank lines should be deleted completely, but single blank lines should be kept.

I know a bit of sed, but the proposed command for removing triple blank lines is over my head:

sed '1N;N;/^\n\n$/d;P;D'

Upvotes: 42

Answers (8)

user17549713

Reputation: 94

As far as I can tell none of the solutions here work. cat -s as suggested by @DerMike isn't POSIX compliant (and it's less convenient if you're already using sed for another transformation), and sed 'N;/^\n$/d;P;D' as suggested by @Birei sometimes deletes more newlines than it should.

Instead, sed ':L;N;s/^\n$//;t L' works. For POSIX compliance use sed -e :L -e N -e 's/^\n$//' -e 't L', since POSIX doesn't specify using ; to separate commands.

Example:

$ S='foo\nbar\n\nbaz\n\n\nqux\n\n\n\nquxx\n';\
> paste <(printf "$S")\
>       <(printf "$S" | sed -e 'N;/^\n$/d;P;D')\
>       <(printf "$S" | sed -e ':L;N;s/^\n$//;t L')
foo     foo     foo
bar     bar     bar
                
baz     baz     baz
        qux     
                qux
qux     quxx    
                quxx
                
                
quxx            
$

Here we can see the original file, @Birei's solution, and my solution side-by-side. @Birei's solution deletes all blank lines separating baz and qux, while my solution removes all but one as intended.

Explanation:

:L        Create a new label called L.

N         Read the next line into the current pattern space,
          separated by an "embedded newline."

s/^\n$//  Replace the pattern space with the empty pattern space,
          corresponding to a single non-embedded newline in the output,
          if the current pattern space only contains a single embedded newline,
          indicating that a blank line was read into the pattern space by `N`
          after a blank line had already been read from the input.

t L       Branch to label L if the previous `s` command successfully
          substituted text in the pattern space.

In effect, this deletes one recurrent blank line at a time, reading each into the pattern space as an embedded newline with N and deleting them with s.

Upvotes: 0

user11130756

Reputation: 11

BUT the above solution only deletes first search of 3 consecutive blank line. To delete all, 3 consecutive blank lines use below command

sed '1N;N;/^\n\n$/ { N;s/^\n\n//;N;D; };P;D' filename

Upvotes: 1

Darsh

Reputation: 1

BUT the above solution only deletes first search of 3 consecutive blank line. To delete all, 3 consecutive blank lines use below command

sed '1N;N;/^\n\n$/ { N;s/^\n\n//;N;D; };P;D' filename

Upvotes: -1

Nestor Urquiza

Reputation: 3027

Just pipe it to 'uniq' command and all empty lines regardless the number of them will be shrank to just one. Simpler is better.

Clarification: As Marlar stated this is not a solution if you have "other non-blank consecutive duplicated lines" that you do not want to get rid of. This is a solution in other cases like when trying to cleanup configuration files which was the solution I was after when I saw this question. I solved my problem indeed just using 'uniq'.

Upvotes: -2

DerMike

Reputation: 16190

This would be easier with cat:

cat -s

Upvotes: 79

Manikandan Rajendran

Reputation: 1182

sed '/^$/{N;/^\n$/d;}'

It will delete only two consecutive blank lines in a file. You can use this expression only in file then only you can fully understand. When a blank line will come that it will enter into braces.

Normally sed will read one line. N will append the second line to pattern space. If that line is empty line. the both lines are separated by newline.

/^\n$/ this pattern will match that time only the d will work. Else d not work. d is used to delete the pattern space whole content then start the next cycle.

Upvotes: 9

Thor

Reputation: 47189

This would be easier with awk:

awk -v RS='\n\n\n' 1

Upvotes: 8

Birei

Reputation: 36282

I've commented the sed command you don't understand:

sed '
    ## In first line: append second line with a newline character between them.
    1N;
    ## Do the same with third line.
    N;
    ## When found three consecutive blank lines, delete them. 
    ## Here there are two newlines but you have to count one more deleted with last "D" command.
    /^\n\n$/d;
    ## The combo "P+D+N" simulates a FIFO, "P+D" prints and deletes from one side while "N" appends
    ## a line from the other side.
    P;
    D
'

Remove 1N because we need only two lines in the 'stack' and it's enought with the second N, and change /^\n\n$/d; to /^\n$/d; to delete all two consecutive blank lines.

A test:

Content of infile:

Run the sed command:

sed '
    N;
    /^\n$/d;
    P;
    D
' infile

That yields:

Upvotes: 20

How to use sed to remove only double empty lines?

Answers (8)

Related Questions