CODEWITHSUNDEEP
bashsed
Itération 122442
Itération 122442

Reputation: 2962

Use a variable as replacement in bash sed command

I am using the sed command on Ubuntu to replace content.

This initial command comes from here.

sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"

However, as you can see, I have a slash in the replacement. The slash causes the command to throw:

sed: -e expression #1, char 9: unknown option to `s'

Moreover, my replacement is stored in a variable.

So the following will not work because of the slash:

sed -i "$ s/$/ $1/" "$DIR./result/doc.md"

As stated here and in duplicate, I should use another delimiter. If I try with @:

   sed -i "$ s@$@ $1@" "$DIR./result/doc.md"

It gives the error:

sed: -e expression #1, char 42: unterminated `s' command

My question is:

How can I use a variable in this command as well as other delimiter than / ?

Upvotes: 1

Views: 1425

Answers (3)

glenn jackman
glenn jackman

Reputation: 246744

Use shell parameter expansion to add escapes to the slashes in the variable:

$ cat file
foo
bar
baz

$ set -- ' /repl'

$ sed "s/$/$1/" file
sed: 1: "s/$/ /repl/": bad flag in substitute command: 'r'

$ sed "s/$/${1//\//\\\/}/" file
foo /repl
bar /repl
baz /repl

That is a monstrosity of leaning toothpicks, but it serves to transform this:

sed "s/$/ /repl/"

into

sed "s/$/ \/repl/"

The same technique can be used for whatever you choose as the sed s/// delimiter.

Upvotes: 1

Charles Duffy
Charles Duffy

Reputation: 295272

Don't use sed here; perl and awk allow more robust approaches.

sed doesn't allow variables to be passed out-of-band from code, so they always need to be escaped. Use a language without that limitation, and you have code that always works, no matter what characters your data contains.

The Short Answer: Using perl

The below is taken from BashFAQ #21:

inplace_replace() {
  local search=$1; shift; local replace=$1; shift
  in="$search" out="$replace" perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' "$@"
}
inplace_replace '@' "replacement" "$DIR/result/doc.md"

The Longer Answer: Using awk

...or, using awk to do a streaming replacement, and a shell function to make that file replacement instead:

# usage as in: echo "in should instead be out" | gsub_literal "in" "out"
gsub_literal() {
  local search=$1 replace=$2
  awk -v s="${search//\\/\\\\}" -v r="${rep//\\/\\\\}" 'BEGIN {l=length(s)} {o="";while (i=index($0, s)) {o=o substr($0,1,i-1) r; $0=substr($0,i+l)} print o $0}'
}

# usage as in: inplace_replace "in" "out" /path/to/file1 /path/to/file2 ...
inplace_replace() {
  local search=$1 replace=$2 retval=0; shift; shift
  for file; do
    tempfile=$(mktemp "$file.XXXXXX") || { retval |= $?; continue; }
    if gsub_literal "$search" "$replace" <"$file" >"$tempfile"; then
      mv -- "$tempfile" "$file" || (( retval |= $? ))
    else
      rm -f -- "$tempfile" || (( retval |= $? ))
    fi
  done
}

Upvotes: 1

Mark
Mark

Reputation: 4455

TL;DR:

Try:

sed -i '$ s@$@ '"$1"'@' "$DIR./result/doc.md"

Long version:

Let's start with your original code:

sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"

And let's compare it to the code you referenced:

sed -i '$ s/$/abc/' file.txt

We can see that they don't exactly match up. I see that you've correctly made this substitution:

file.txt --> "$DIR./result/doc.md"

That looks fine (although I do have my doubts about the . after $DIR ). However, the other substitution doesn't look great:

abc -->  /replacement

You actually introduced another delimeter /. However, if we replace the delimiters with '@' we get this:

sed -i '$ s@$@ /replacement@' "$DIR./result/doc.md"

I think that the above is perfectly valid in sed/bash. The $@ will not be replaced by the shell because it is single quoted. The $DIR variable will be interpolated by the shell because it is double quoted.

Looking at one of your attempts:

sed -i "$ s@$@ $1@" "$DIR./result/doc.md"

You will have problems due to the shell interpolation of $@ in the double quotes. Let's correct that by replacing with single quotes (but leaving $1 unquoted):

sed -i '$ s@$@ '"$1"'@' "$DIR./result/doc.md"

Notice the '"$1"'. I had to surround $1 with '' to basically unescape the surrounding single quotes. But then I surrounded the $1 with double quotes so we could protect the string from white spaces.

Upvotes: 1

Related Questions