CODEWITHSUNDEEP
bashsedfreebsd
Allan
Allan

Reputation: 1272

Insert newline (\n) using sed

I am trying to scrub some lists into a properly formatted CSV file for database import.

My starting file, looks something like this with what is supposed to be each "line" spanning multiple lines like below

Mr. John Doe
Exclusively Stuff, 186 
Caravelle Drive, Ponte Vedra
33487.

I created a sed script that cleans up the file (there's lots of "dirty" formatting like double spaces and spaces before/after commas). The problem is the Zip with the period. I would like to change that period for a new line, but I cannot get it to work.

The command that I use is:

sed -E -f scrub.sed test.txt

and the scrub.sed script is as follows:

:a
N
s|[[:space:]][[:space:]]| |g
s|,[[:space:]]|,|g
s|[[:space:]],|,|g
s|\n| |g
s|[[:space:]]([0-9]{5})\.|,FL,\1\n |g
$!ba

What I get is

Mr. John Doe,Exclusively Stuff,186 Caravelle Drive,Ponte Vedra,FL,33487n

If figured that the Zip+.(period) would be a great "delimiter" to use the substitution on and while I can find it, I can't seem to tell it to put a newline there.

Most of the things I found online are about replacing the newline with something else (usually deleting them), but not much on replacing with a newline. I did find this, but it didn't work: How to insert newline character after comma in `),(` with sed?

Is there something I am missing?

Update:

I edited my scrub.sed file putting the literal new line as instucted. It still doesn't work

:a
N
s|[[:space:]][[:space:]]| |g
s|,[[:space:]]|,|g
s|[[:space:]],|,|g
s|\n| |g
s|[[:space:]]([0-9]{5})\.|,FL,\1\
|g
$!ba

What I get is (everything on one line):

Mr. John Doe,Exclusively Stuff,186 Caravelle Drive,Ponte Vedra,FL,33487 Mrs. Jane Smith,Props and Stuff,123 Main Drive,Jacksonville,FL,336907

My expected output should be:

Mr. John Doe,Exclusively Stuff,186 Caravelle Drive,Ponte Vedra,FL,33487
Mrs. Jane Smith,Props and Stuff,123 Main Drive,Jacksonville,FL,336907

Upvotes: 95

Views: 206633

Answers (4)

dawg
dawg

Reputation: 103694

The sed on BSD does not support the \n representation of a new line (turning it into a literal n):

$ echo "123." | sed -E 's/([[:digit:]]*)\./\1\n next line/'
123n next line

GNU sed does support the \n representation:

$ echo "123." | gsed -E 's/([[:digit:]]*)\./\1\nnext line/'
123
next line

Alternatives are:

Use a single character delimiter that you then use tr translate into a new line:

$ echo "123." | sed -E 's/([[:digit:]]*)\./\1|next line/' | tr '|' '\n'
123
next line

Or use an escaped literal new line in your sed script:

$ echo "123." | sed -E 's/([[:digit:]]*)\./\1\
next line/'
123
next line

Or define a new line:

POSIX:

nl='
'

BASH / zsh / others that support ANSI C quoting:

nl=$'\n'

And then use sed with appropriate quoting and escapes to insert the literal \n:

echo "123." | sed 's/\./'"\\${nl}"'next line/'
123
next line

Or use awk:

$ echo "123." | awk '/^[[:digit:]]+\./{sub(/\./,"\nnext line")} 1'
123
next line

Or use GNU sed which supports \n

Upvotes: 129

Mohasin Ali
Mohasin Ali

Reputation: 4015

Add a line after a match.

The sed command can add a new line after a pattern match is found. The "a" command to sed tells it to add a new line after a match is found.

sed '/unix/ a "Add a new line"' file.txt

unix is great os. unix is opensource. unix is free os.

    "Add a new line"
    
    learn operating system.
    
    unixlinux which one you choose.
    
    "Add a new line"

Add a line before a match

The sed command can add a new line before a pattern match is found. The "i" command to sed tells it to add a new line before a match is found.

sed '/unix/ i "Add a new line"' file.txt

"Add a new line"

unix is great os. unix is opensource. unix is free os.

learn operating system.

"Add a new line"

unixlinux which one you choose.

Upvotes: 3

interestedparty333
interestedparty333

Reputation: 2536

The following works on Oracle Linux, x8664:

$ echo 'foobar' | sed 's/foo/foo\n/'
foo
bar

If you need it to match more than once per line, you'll need to place a g at the end, as in:

$ echo 'foobarfoobaz' | sed 's/foo/foo\n/g'
foo
barfoo
baz

Upvotes: 3

Ed Morton
Ed Morton

Reputation: 203129

The portable way to get a newline in sed is a backslash followed by a literal newline:

$ echo 'foo' | sed 's/foo/foo\
bar/'
foo
bar

I guarantee there's a far simpler solution to your whole problem by using awk rather than sed though.

Upvotes: 17

Related Questions