CODEWITHSUNDEEP
linuxunixsed
jonas
jonas

Reputation: 5509

Delete empty lines using sed

I am trying to delete empty lines using sed:

sed '/^$/d'

but I have no luck with it.

For example, I have these lines:

xxxxxx


yyyyyy


zzzzzz

and I want it to be like:

xxxxxx
yyyyyy
zzzzzz

What should be the code for this?

Upvotes: 521

Views: 738311

Answers (16)

fedorqui
fedorqui

Reputation: 290445

I am missing the awk solution:

awk 'NF' file

Which would return:

xxxxxx
yyyyyy
zzzzzz

How does this work? Since NF stands for "number of fields", those lines being empty have 0 fields, so that awk evaluates 0 to False and no line is printed; however, if there is at least one field, the evaluation is True and makes awk perform its default action: print the current line.

Upvotes: 216

Kpym
Kpym

Reputation: 4033

If you want to use modern Rust tools, you can consider:

  • ripgrep:
    • cat datafile | rg '.' line with spaces is considered non empty
    • cat datafile | rg '\S' line with spaces is considered empty
    • rg '\S' datafile line with spaces is considered empty (-N can be added to remove line numbers for on screen display)
  • sd
    • cat datafile | sd '^\n' '' line with spaces is considered non empty
    • cat datafile | sd '^\s*\n' '' line with spaces is considered empty
    • sd '^\s*\n' '' datafile inplace edit

Upvotes: 2

Samuel Kenneth
Samuel Kenneth

Reputation: 109

The command you are trying is correct, just use -E flag with it.

sed -E '/^$/d'

-E flag makes sed catch extended regular expressions. More info here

Upvotes: 9

Nilesh Shukla
Nilesh Shukla

Reputation: 11

Using vim editor to remove empty lines

:%s/^$\n//g

Upvotes: -1

user319660
user319660

Reputation: 322

Another option without sed, awk, perl, etc

strings $file > $output

strings - print the strings of printable characters in files.

Upvotes: 19

Oleg Mazko
Oleg Mazko

Reputation: 1830

sed

grep

awk

Upvotes: 105

Kent
Kent

Reputation: 195259

You may have spaces or tabs in your "empty" line. Use POSIX classes with sed to remove all lines containing only whitespace:

sed '/^[[:space:]]*$/d'

A shorter version that uses ERE, for example with gnu sed:

sed -r '/^\s*$/d'

(Note that sed does NOT support PCRE.)

Upvotes: 841

Douglas Daseeco
Douglas Daseeco

Reputation: 3671

You are most likely seeing the unexpected behavior because your text file was created on Windows, so the end of line sequence is \r\n. You can use dos2unix to convert it to a UNIX style text file before running sed or use

sed -r "/^\r?$/d"

to remove blank lines whether or not the carriage return is there.

Upvotes: 4

justincbagley
justincbagley

Reputation: 11

My bash-specific answer is to recommend using perl substitution operator with the global pattern g flag for this, as follows:

$ perl -pe s'/^\n|^[\ ]*\n//g' $file
xxxxxx
yyyyyy
zzzzzz

This answer illustrates accounting for whether or not the empty lines have spaces in them ([\ ]*), as well as using | to separate multiple search terms/fields. Tested on macOS High Sierra and CentOS 6/7.

FYI, the OP's original code sed '/^$/d' $file works just fine in bash Terminal on macOS High Sierra and CentOS 6/7 Linux at a high-performance supercomputing cluster.

Upvotes: 1

Vitaly
Vitaly

Reputation: 15

For me with FreeBSD 10.1 with sed worked only this solution: 

sed -e '/^[     ]*$/d' "testfile"

inside [] there are space and tab symbols.

test file contains:

fffffff next 1 tabline ffffffffffff

ffffffff next 1 Space line ffffffffffff

ffffffff empty 1 lines ffffffffffff

============ EOF =============

Upvotes: -2

Claes Wikner
Claes Wikner

Reputation: 1517

This works in awk as well.

awk '!/^$/' file
xxxxxx
yyyyyy
zzzzzz

Upvotes: 3

tank
tank

Reputation: 61

You can say:

sed -n '/ / p' filename    #there is a space between '//'

Upvotes: 4

Vadim
Vadim

Reputation: 849

I believe this is the easiest and fastest one:

cat file.txt | grep .

If you need to ignore all white-space lines as well then try this:

cat file.txt | grep '\S'

Example:

s="\
\
a\
 b\
\
Below is TAB:\
    \
Below is space:\
 \
c\
\
"; echo "$s" | grep . | wc -l; echo "$s" | grep '\S' | wc -l

outputs

7
5

Upvotes: 47

Lowbit
Lowbit

Reputation: 21

You can do something like that using "grep", too:

egrep -v "^$" file.txt

Upvotes: 2

ConMan
ConMan

Reputation: 1652

With help from the accepted answer here and the accepted answer above, I have used: 

$ sed 's/^ *//; s/ *$//; /^$/d; /^\s*$/d' file.txt > output.txt

`s/^ *//`  => left trim
`s/ *$//`  => right trim
`/^$/d`    => remove empty line
`/^\s*$/d` => delete lines which may contain white space

This covers all the bases and works perfectly for my needs. Kudos to the original posters @Kent and @kev

Upvotes: 17

Alberto Zaccagni
Alberto Zaccagni

Reputation: 31590

sed '/^$/d' should be fine, are you expecting to modify the file in place? If so you should use the -i flag.

Maybe those lines are not empty, so if that's the case, look at this question Remove empty lines from txtfiles, remove spaces from start and end of line I believe that's what you're trying to achieve.

Upvotes: 87

Related Questions