CODEWITHSUNDEEP
linuxbashshlsquoting
choroba
choroba

Reputation: 241898

Newlines not quoted properly in ls -Q

Using ls -Q with --quoting-style=shell, newlines in file names (yes, I know...) are turned into ?. Is this a bug? Is there a way how to get the file names in a format that's 100% compatible with a shell (sh or bash if possible)?

Example (bash):

$ touch a$'\n'b
$ for s in literal shell shell-always c c-maybe escape locale clocale ; do
      ls -Q a?b --quoting-style=$s
  done
a?b
'a?b'
'a?b'
"a\nb"
"a\nb"
a\nb
‘a\nb’
‘a\nb’

Upvotes: 2

Views: 268

Answers (3)

pixelbeat
pixelbeat

Reputation: 31728

coreutils 8.25 has the new 'shell-escape' quoting style, and in fact enables it by default to allow the output from ls to be always usable, and to be safe to copy and paste back to other commands.

Upvotes: 2

ruakh
ruakh

Reputation: 183321

From a bit of experimentation, it looks like --quoting-style=escape is compatible with being wrapped in $'...', with two exceptions:

  • it escapes spaces by prepending a backslash; but $'...' doesn't discard backslashes before spaces.
  • it doesn't escape single-quotes.

So you could perhaps write something like this (in Bash):

function ls-quote-shell () {
    ls -Q --quoting-style=escape "$@" \
    | while IFS= read -r filename ; do
        filename="${filename//'\ '/ }"  # unescape spaces
        filename="${filename//"'"/\'}"  # escape single-quotes
        printf "$'%s'\n" "$filename"
      done
}

To test this, I've created a directory with a bunch of filenames with weird characters; and

eval ls -l $(ls-quote-shell)

worked as intended . . . though I won't make any firm guarantees about it.

Alternatively, here's a version that uses printf to process the escapes followed by printf %q to re-escape in a shell-friendly manner:

function ls-quote-shell () {
    ls -Q --quoting-style=escape "$@" \
    | while IFS= read -r escaped_filename ; do
        escaped_filename="${escaped_filename//'\ '/ }"  # unescape spaces
        escaped_filename="${escaped_filename//'%'/%%}"  # escape percent signs
        # note: need to save in variable, rather than using command
        # substitution, because command substitution strips trailing newlines:
        printf -v filename "$escaped_filename"
        printf '%q\n' "$filename"
      done
}

but if it turns out that there's some case that the first version doesn't handle correctly, then the second version will most likely have the same issue. (FWIW, eval ls -l $(ls-quote-shell) worked as intended with both versions.)

Upvotes: 0

chepner
chepner

Reputation: 531205

Maybe not quite what you are looking for, but the "escape" style seems to work well with the upcoming ${...@E} parameter expansion in bash 4.4.

$ touch $'a\nb' $'c\nd'
$ ls -Q --quoting-style=escape ??? | while IFS= read -r fname; do echo =="${fname@E}==="; done
==a
b==
==c
d==

Here is the relevant part of the man page (link is to the raw source):

${parameter@operator}
          Parameter transformation.  The expansion is either a transforma-
          tion  of  the  value of parameter or information about parameter
          itself, depending on the value of operator.  Each operator is  a
          single letter:

          Q      The  expansion is a string that is the value of parameter
                 quoted in a format that can be reused as input.
          E      The expansion is a string that is the value of  parameter
                 with  backslash  escape  sequences  expanded  as with the
                 $'...' quoting mechansim.
          P      The expansion is a string that is the result of expanding
                 the value of parameter as if it were a prompt string (see
                 PROMPTING below).
          A      The expansion is a string in the form  of  an  assignment
                 statement  or  declare  command  that, if evaluated, will
                 recreate parameter with its attributes and value.
          a      The expansion is a string consisting of flag values  rep-
                 resenting parameter's attributes.

          If  parameter  is @ or *, the operation is applied to each posi-
          tional parameter in turn, and the  expansion  is  the  resultant
          list.   If  parameter is an array variable subscripted with @ or
          *, the case modification operation is applied to each member  of
          the array in turn, and the expansion is the resultant list.

          The  result  of  the  expansion is subject to word splitting and
          pathname expansion as described below.

Upvotes: 0

Related Questions