CODEWITHSUNDEEP
bashsedsymbolsreplace
Village
Village

Reputation: 24413

How to find and replace the first match of a BASH variable in a file, when the replacement contains backslashes?

I am trying to do a find and replace with sed and BASH variables. As the replaced content contains many / characters, I have used # instead of /, as is typically used in sed:

 sed -i "0,\#^$word#s##$replacement#" ./file

The item I want to replace contains the \ symbol:

 replacement=$(echo "\macro{30\percent of bears eat fish.}")

However, when I do the replacement, \ always disappears.

How can I do a find and replace with sed when \, /, or other symbols might be found within the replacement text?

Upvotes: 1

Views: 1519

Answers (6)

wich
wich

Reputation: 17157

word=$(echo -E "wo\rd")
replacement=$(echo -E "r/e\p")

echo -ne "a\nb\nc\nd\nwo\\\\rd\ne\nf\n" | sed -e "0,\#^${word//\\/\\\\}# s##${replacement//\\/\\\\}#"

Note that you will still have problems with other characters, & and # in replacement string and of course all valid regex characters in the word string

The below takes care of the regular expression chars

word=$(echo -E "^\$.*[#]wo\rd" | sed 's:[]\[\^\$\.\*#\\]:\\&:g')
replacement=$(echo -E "&#r/e\p" | sed 's:[&#\\]:\\&:g')

echo -ne "a\nb\nc\nd\n^\$.*[#]wo\\\\rd\ne\nf\n" | sed -e "0,\#^${word}# s##${replacement}#"

Upvotes: 2

Debaditya
Debaditya

Reputation: 2497

Try this

Input 

/home/Scripts/dummyrun

Unix Command

$> sed -i "s%/home/Scripts/dummyrun%Newpath%g" Input

Output

Open the Input file and changes will be reflected automatically.

 $> cat Input 
    Newpath

The above command replaces /home/Scripts/dummyrun to Newpath in Input file.

Upvotes: 1

ghoti
ghoti

Reputation: 46856

Your problem may not be the backslashes. Or at least, one of your problems. :-)

In the sed installed on my system, you can use # as the delimiter of your substitution (i.e. s#pattern#replacement#), but not as part of your initial search. Thus, you would have to use something like:

 sed -i "1,/^$word/s##$replacement#" ./file

See my results:

[ghoti@pc ~]$ printf 'one\ntwo\nthree\nfour\nfive\n' | sed '1,/three/d'
four
five
[ghoti@pc ~]$ printf 'one\ntwo\nthree\nfour\nfive\n' | sed '1,#three#d'
sed: 1: "1,#three#d": expected context address
[ghoti@pc ~]$

I'm in FreeBSD, but I'm pretty sure the GNU sed works the same way.

This doesn't help you if there are incompatible characters in the pattern you're trying to match, though. You just need to make sure that $word is a properly formatted regular expression that can be used within sed.

As for the backslashes ... if the advice above gives you any sed joy, try just doubling any backslashes in the replacement pattern. But work on just one problem at a time, and start with simpler $replacements to make sure your basic sed notation is functioning.

UPDATE:

Also, for the fun of it:

[ghoti@pc ~]$ word="word"
[ghoti@pc ~]$ replacement="\\\macro{some text};"
[ghoti@pc ~]$ printf "Replace a word with some text.\n" | sed "s#$word#$replacement#"
Replace a \macro{some text}; with some text.
[ghoti@pc ~]$ replacement="\\\\macro{some text};"
[ghoti@pc ~]$ printf "Replace a word with some text.\n" | sed "s#$word#$replacement#"
Replace a \macro{some text}; with some text.
[ghoti@pc ~]$ printf "Replace a word with some text.\n" | sed "s/$word/$replacement/"
Replace a \macro{some text}; with some text.
[ghoti@pc ~]$ word="two"
[ghoti@pc ~]$ printf "one\ntwo\nthree\n" | sed "s/^$word/$replacement/"
one
\macro{some text};
three
[ghoti@pc ~]$

Upvotes: 4

Jonathan Leffler
Jonathan Leffler

Reputation: 754280

Backslashes make life complicated

Life always gets tricky when you have the shell and sed both trying to interpret backslashes. In fact, it gets really painful.

Let's take a simple case: you want sed to replace the first backslash on a line by an at sign:

sed 's/\\/@/' file

Note that we needed to specify two backslashes here, and because the command was in single quotes, the shell didn't do any interpretation for us — thank goodness!

Now, let's suppose we need to use double quotes instead of single quotes. Now we need to type:

sed "s/\\\\/@/" file

Why? Because the shell first scans the string, and it sees backslash-backslash and translates that to a single backslash; and then it sees a second backslash-backslash and translates that to another single backslash. So, then sed gets to see the two backslashes and does what you wanted it to do.

bash-as-bash behaves differently from bash-as-sh

You may have a /bin/echo on your machine (I do on mine, which is Mac OS X 10.7.3) which behaves differently from bash's built-in echo. I can get the following output (from /bin/bash):

$ echo '\\' | sed "s/\\\\/@/"
@\
$ /bin/echo '\\' | sed "s/\\\\/@/"
@\
$ echo '\\' | sed 's/\\\\/@/'
@
$ /bin/echo '\\' | sed 's/\\\\/@/'
@
$

Let's throw another spanner in the works (my brain's hurting — how's yours coping?). I get a different answer from /bin/sh (which is very similar but not identical to /bin/bash; the difference is 64 bytes on disk, and both are bash version 3.2):

$ echo '\\' | sed "s/\\\\/@/"
@
$ /bin/echo '\\' | sed "s/\\\\/@/"
@\
$ echo '\\' | sed 's/\\\\/@/'
\
$ /bin/echo '\\' | sed 's/\\\\/@/'
@
$

Yes, there's a difference. And now I'm confused!

In your discussion, you mention using:

replacement=$(echo "\\\macro{some text}")

This complicates life. You've got the shell processing that text, and bash's built-in echo also processing the text. And if you use bash as sh you get one answer and if you use bash as bash you get another answer.

When I tested with bash as sh, I found:

replacement="\\\\macro{some text}"   # 4 slashes, not 3

This gets two backslashes into $replacement. You can demonstrate that by running:

$ /bin/echo "$replacement"
\\macro{some text}
$ echo "$replacement"
\macro{some text}
$

So helpful...so, to get the $(echo ...) to produce two backslashes in the output when you subsequently echo it, you need no fewer than 16 (!) backslashes in the string:

$ replacement=$(echo "\\\\\\\\\\\\\\\\macro{some text}")
$ echo "$replacement"
\\macro{some text}
$

Beware: that applies to bash-as-sh; running bash-as-bash, you get:

$ replacement=$(echo "\\\\\\\\\\\\\\\\macro{some text}")
$ echo "$replacement"
\\\\\\\\macro{some text
$

Isn't life fun! So, to get this to work, you're going to have to preprocess your search pattern and your replacement patterns rather carefully to get enough backslashes into each one. How many? Ugh...

Test script

Here's a test script which I called, with stunning originality and explicitness of name, xx:

#!/bin/bash

echo "happy /word/ today" > file

word="/word/"
/bin/echo "word=<<$word>>"

replacement='\\\\macro{some text}'
/bin/echo "repl=<<$replacement>>"
echo "repl=<<$replacement>>"

/bin/echo sed -e "s#$word#$replacement#" file
sed -e "s#$word#$replacement#" file

echo
replacement="\\\\macro{some text}"
/bin/echo "repl=<<$replacement>>"
echo "repl=<<$replacement>>"

/bin/echo sed -e "s#$word#$replacement#" file
sed -e "s#$word#$replacement#" file


echo
replacement=$(echo "\\\\\\\\macro{some text}")
/bin/echo "$replacement"
echo "$replacement"
echo

replacement=$(echo "\\\\\\\\\\\\\\\\macro{some text}")
/bin/echo "$replacement"
echo "$replacement"

Test script run by bash-as-bash

This is the output from bash xx:

$ bash xx
word=<</word/>>
repl=<<\\\\macro{some text}>>
repl=<<\\\\macro{some text}>>
sed -e s#/word/#\\\\macro{some text}# file
happy \\macro{some text} today

repl=<<\\macro{some text}>>
repl=<<\\macro{some text}>>
sed -e s#/word/#\\macro{some text}# file
happy \macro{some text} today

\\\\macro{some text}
\\\\macro{some text}

\\\\\\\\macro{some text}
\\\\\\\\macro{some text}
$

Test script run by bash-as-sh

And this is the output from sh xx:

$ sh xx
word=<</word/>>
repl=<<\\\\macro{some text}>>
repl=<<\\macro{some text}>>
sed -e s#/word/#\\\\macro{some text}# file
happy \\macro{some text} today

repl=<<\\macro{some text}>>
repl=<<\macro{some text}>>
sed -e s#/word/#\\macro{some text}# file
happy \macro{some text} today

\\macro{some text}
\macro{some text}

\\\\macro{some text}
\\macro{some text}
$

Somewhere in amongst all that is (most of) the answer to your problem. To say I was surprised to find this much difference between bash-as-bash and bash-as-sh doesn't begin to cover the territory.

Upvotes: 5

potong
potong

Reputation: 58463

This might work for you:

sed -i "0,\#^$word#s##$(printf '%q' $replacement)#" ./file

Upvotes: 2

Zsolt Botykai
Zsolt Botykai

Reputation: 51633

Try this (if I understand your problem correctly):

R="macro{some text}"
sed -i "0,\#^$word#s##\\$R#" ./file

Upvotes: 1

Related Questions