bash - quickly escape any characters string

Question

I'm looking for a utility that can quickly escape a string of characters. This task is so useful but i can't found it.

Lets say for an example:

hisrmline 'h | g -E "^ [0-9]*  exit$"'

If i want to manually escape it, it can be done:

'hisrmline '\''h | g -E "^ [0-9]*  exit$"'\'''

But it's time consuming and not efficient. So i found printf %q:

[xiaobai@xiaobai note]$ printf "%q" hisrmline 'h | g -E "^ [0-9]*  exit$"'
hisrmlineh\ \|\ g\ -E\ \"\^\ \[0-9\]\*\ \ exit\$\"[xiaobai@xiaobai note]$ 
[xiaobai@xiaobai note]$ 

The output is wrong because hisrmlineh was concate together, so i'm narrow down the strings:

[xiaobai@xiaobai note]$ printf "%q" hisrmline 'h'
hisrmlineh[xiaobai@xiaobai note]$ 
[xiaobai@xiaobai note]$ 

What i desired was hisrmline\ \'h\'

This is especially useful on grep:

[xiaobai@xiaobai note]$ HISTTIMEFORMAT=""; history|grep -a --color=auto hisrmline\ \'h
 7856  hisrmline 'hisrmline'
 7857  hisrmline 'hisrmline'
 7882  hisrmline 'h | g -E "^ [0-9]*  exit[ ]*$"'
 7883  hisrmline 'h | g -E "^ [0-9]*  exit[ ]*$"'
 7884  hisrmline 'h | g -E "'
 7885  hisrmline 'h | g '
 7886  hisrmline 'h | g'
 7887  hisrmline 'h |'
 7890  hisrmline 'h | g -E "^ [0-9]*  exit$"'
 7891  hisrmline 'h | g -E "^ [0-9]*  exit$"'
 7905  h|g 'hisrmline 'h | g -E "^ [0-9]*  exit$"''

And grep -F wouldn't make my life easier when deal with nested single quote, i still have to manually escape the single quote '\'':

[xiaobai@xiaobai note]$ HISTTIMEFORMAT=""; history|grep -a --color=auto -F  '[0-9]*  exit$"'\'''
 7889  h|g -aF 'h | g -E "^ [0-9]*  exit$"'
 7890  hisrmline 'h | g -E "^ [0-9]*  exit$"'
 7891  hisrmline 'h | g -E "^ [0-9]*  exit$"'
 7905  h|g 'hisrmline 'h | g -E "^ [0-9]*  exit$"''
 7911  h|g 'hisrmline 'h | g -E "^ [0-9]*  exit$"''
 7912  h|g 'hisrmline '"'"'h | g -E "^ [0-9]*  exit$"'"'"'

Is there any easier way or any existing utility to escape list of any characters string ?

Answer 1

If you quote the command line properly then printf should work e.g.:

printf "%q\n" "hisrmline 'h'"
hisrmline\ \'h\'

Or:

printf "%q\n" "hisrmline 'h | g -E \"^ [0-9]*  exit$\"'"
hisrmline\ \'h\ \|\ g\ -E\ \"\^\ \[0-9\]\*\ \ exit\$\"\'

---

EDIT: You're probably looking for this kind of escaping:

IFS= read -r str<<"EOF"
hisrmline 'h | g -E "^ [0-9]*  exit$"'
EOF

printf "%q\n" "$str"
hisrmline\ \'h\ \|\ g\ -E\ \"\^\ \[0-9\]\*\ \ exit\$\"\'

[UPDATE by @林果皞]

To whom might be interested, EOF have to quoted to prevent expansion, as noted by @bize:

EOF without quoted:

[xiaobai@xiaobai Downloads]$ IFS= read -r str<<EOF
> target='h | g -E -i -e "^[ ]+[0-9]+  .*[|&; ]+g[ ]" -e "^[ ]+[0-9]+  .*[|&; ]+g$"'; history|grep -aF "$target"; echo ${#target}
> EOF
[xiaobai@xiaobai Downloads]$ printf "%q\n" "$str"
target=\'h\ \|\ g\ -E\ -i\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\[\ \]\"\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\$\"\'\;\ history\|grep\ -aF\ \"h\ \|\ g\ -E\ -i\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\[\ \]\"\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\$\"\"\;\ echo\ 73
[xiaobai@xiaobai Downloads]$ 

"EOF" quoted:

[xiaobai@xiaobai Downloads]$ IFS= read -r str<<"EOF"
> target='h | g -E -i -e "^[ ]+[0-9]+  .*[|&; ]+g[ ]" -e "^[ ]+[0-9]+  .*[|&; ]+g$"'; history|grep -aF "$target"; echo ${#target}
> EOF
[xiaobai@xiaobai Downloads]$ printf "%q\n" "$str"
target=\'h\ \|\ g\ -E\ -i\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\[\ \]\"\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\$\"\'\;\ history\|grep\ -aF\ \"\$target\"\;\ echo\ \$\{#target\}
[xiaobai@xiaobai Downloads]$ 

Correct behaviour only when supplied from the output of quoted "EOF":

[xiaobai@xiaobai Downloads]$ h|g -F target=\'h\ \|\ g\ -E\ -i\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\[\ \]\"\ -e\ \"\^\[\ \]+\[0-9\]+\ \ .\*\[\|\&\;\ \]+g\$\"\'\;\ history\|grep\ -aF\ \"\$target\"\;\ echo\ \$\{#target\}
 7721  target='h | g -E -i -e "^[ ]+[0-9]+  .*[|&; ]+g[ ]" -e "^[ ]+[0-9]+  .*[|&; ]+g$"'; history|grep -aF "$target"; echo ${#target}
 7725  target='h | g -E -i -e "^[ ]+[0-9]+  .*[|&; ]+g[ ]" -e "^[ ]+[0-9]+  .*[|&; ]+g$"'; history|grep -aF "$target"; echo ${#target}
 7726  atarget='h | g -E -i -e "^[ ]+[0-9]+  .*[|&; ]+g[ ]" -e "^[ ]+[0-9]+  .*[|&; ]+g$"'; history|grep -aF "$target"; echo ${#target}
 8297  target='h | g -E -i -e "^[ ]+[0-9]+  .*[|&; ]+g[ ]" -e "^[ ]+[0-9]+  .*[|&; ]+g$"'; history|grep -aF "$target"; echo ${#target}
 8320  target='h | g -E -i -e "^[ ]+[0-9]+  .*[|&; ]+g[ ]" -e "^[ ]+[0-9]+  .*[|&; ]+g$"'; history|grep -aF "$target"; echo ${#target}

*h is aliased to export HISTTIMEFORMAT=""; history *g is aliased togrep -a --color=auto

Direct use of $ h|g -F "$str" is working too.

When deal with unicode, i have to assign LC_ALL= to empty(i.e LC_ALL="en_US.utf8") before query(history, ls..etc) the source string. Then i have to switch it to LC_ALL=C to make printf %q working correctly.

Answer 2

A kiss aproach:

printf "%q" "$(cat <<"_up_to_here_"
hisrmline 'h | g -E "^ [0-9]*  exit$"'
_up_to_here_
)"

Anything betwen "_up_to_here_" and _up_to_here_ will be correctly quoted.

PLEASE NOTE: The first _up_to_here_ is quoted to prevent expansion of any $variable in the next line or line(S).

COMMENT: the use of cat is intended to keep the command simple, any attemp to correctly convert to read will require extensive knowledge: Not a KISS aproach.

Answer 3

Update:

In comments you told that you copy the lines from history and want to reinsert them in a shell command. In bash there is history expansion to access parts of the history or modify it. Probably this is already what you want.

Otherwise you could create a little command to display the history escaped:

IFS=$'\n' history | while read line ; do printf "%q\n" "$line"; done

You can copy lines from that output and insert them into a shell string. You may additionally pipe this to less if your $HISTSIZE is large.

If you need this command more often you may create a script file out of it or create a function in the .bashrc

---

Original Answer

Let's say want to use the ' as the string delimiter, you can use the following bash expression:

string="hisrmline 'h | g -E \"^ [0-9]*  exit$\"'"
echo "${string//\'/\\\'}"

Now you can use the string in bash. If you want to use it in grep or other programs which use regular expression, you would need to escape further characters. However grep supports the option -F. If you pass it, patterns are handled as fixed strings, not as regular expressions.