Python3: shell quoting less complicated than shlex.quote() output?

Question

In python3.8, I have this code:

import shlex
item = "ABC'DEF"
quoteditem = shlex.quote(item)
print(quoteditem)

This is the output:

'ABC'"'"'DEF'

It's difficult to discern the double and single quotes here on this web page, so this is a description of what is printed:

single-quote
ABC
single-quote
double-quote
single-quote
double-quote
single-quote
DEF
single-quote

This is, of course, a correct shell quoting, but it is not the only possible shell quoting, and it is overly complex.

Another possibility is simply this:

"ABC'DEF"

And here's a second possibility:

ABC\'DEF

I much prefer these simpler versions. I know how to write python code to convert the complicated version into one of these simpler forms, but I'm wondering if there might be an already existing python function which can perform this kind of simpler shell quoting.

Thank you in advance for any suggestions.

Answer 1

shlex.quote is optimized for performance, so the output is machine-readable, but less pretty for human eyes. producing a "pretty" string is slower

when you wrap the string in double-quotes (for a shell), you also need to escape expressions like $x or $(x) or !x

import re

_shell_quote_is_unsafe = re.compile(r'[^\w@%+=:,./-]', re.ASCII).search
# note: [^...] = negated char class -> safe chars are \w@%+=:,./-

_shell_quote_replace = re.compile(r'([\\$"!])', re.ASCII).sub

def shell_quote(s: str):
    """
    Return a shell-escaped version of the string *s*.
    Wrap string in double-quotes when necessary.
    Based on shlex.quote (wrap string in single-quotes)
    """
    if not s:
        return '""'
    if _shell_quote_is_unsafe(s) is None:
        return s
    return '"' + _shell_quote_replace(r"\\\1", s) + '"'

# test
assert shell_quote('a"b\\c$d$(e)!f\tg\nh\ri') == '"a\\"b\\\\c\\$d\\$(e)\\!f\tg\nh\ri"'

Answer 2

This is sort-of an answer. It doesn't provide "... an already existing python function which can perform this kind of simpler shell quoting", as I requested, since it now seems like such a quoting function doesn't exist in python. However, it does show how I coded a quoting mechanism that provides simpler output (for python-3.6 or above):

def shellquote(item):
    if not item:
        return "''"
    # Pre-escape any escape characters                                                                                                                    
    item = item.replace('\\', r'\\')
    if "'" not in item:
        # Contains no single quotes, so we can                                                                                                        
        # single-quote the output.                                                                                                                    
        return f"'{item}'"
    else:
        # Enclose in double quotes. We must escape                                                                                                    
        # "$" and "!", which which normally trigger                                                                                                   
        # expansion in double-quoted strings in shells.                                                                                               
        # If it contains double quotes, escape them, also.                                                                                               
        item = item.replace(r'$', r'\$') \
                   .replace(r'!', r'\!') \
                   .replace(r'"', r'\"')
        return f'"{item}"'

For earlier versions of python which don't support f-strings, format can be used instead of those f-strings.

Here are some examples. The lefthand column shows the assignment statement of the pythonString variable within the python program. The righthand column shows what will then appear on the terminal when print(shellquote(pythonString)) is invoked from within the python program:

pythonString='ABC"DEF'       printed output: 'ABC"DEF'
pythonString="ABC'DEF"       printed output: "ABC'DEF"
pythonString='ABC\'DEF'      printed output: "ABC'DEF"
pythonString="ABC\"DEF"      printed output: 'ABC"DEF'
pythonString='ABC\\"DEF'     printed output: 'ABC\\"DEF'
pythonString="ABC\\'DEF"     printed output: "ABC\\'DEF"
pythonString="AB'C$DEF"      printed output: "AB'C\$DEF"
pythonString='AB\'C$DEF'     printed output: "AB'C\$DEF"
pythonString='AB"C$DEF'      printed output: 'AB"C$DEF'
pythonString="AB\"C$DEF"     printed output: 'AB"C$DEF'
pythonString='A\'B"C$DEF'    printed output: "A'B\"C\$DEF"
pythonString='A\\\'B"C$DEF'  printed output: "A\\'B\"C\$DEF"

This is not the only way that the shell quoting could be performed, but at least the output is simpler than what comes out of shlex.quote in many cases.