How do I remove spaces from an output of an ls command AFTER a certain character?

Question

This is my ls command and post-processing:

ls -l $pwd | tail -n +2 | cut -c1-10,50-999999  |  sed 's/./& /g'  |
    sed  's/\(.\{7\}\)/&  /g' | sed 's/\(.\{30\}\)/&/g'

This is the output:

- r w x   r - -   r - -     a d d   . o l d   

I want to remove all the spaces within the filename, so I can end up with something like this: (keep in mind the space pattern is kept at the permissions)

- r w x   r - -   r - -     add.old   

Answer 1

Writing bullet-proof code to parse ls output is tricky if the file names are not reasonably clean. With the caveat that your file names must not contain newlines and should avoid other control characters (but more or less anything else should be OK) you can just about risk parsing the output from ls. If your file names are limited to the portable file name character set ([-a-zA-Z0-9_.]), you should be fine. But be aware that not everyone will be as disciplined as you are with their file names, so your scripts can fail suddenly if someone creates a name with unexpected characters. Note that leading dashes in file names can wreak havoc; a name such as --version will make the average GNU utility behave unexpectedly, for example.

• You were warned!

You can use cut -c1-10,50- to avoid typing all those 9's. You can combine your 3 sed commands into one using -e in front of each expression.

With a little bit of sed trickery, you can deal with the spacing more easily. On my Mac, the correct column for the start of the file name is column 54, not 50:

$ ls -l |
> sed -e '1d;12q' |     # Only 11 file names listed
> cut -c 1-10,54- |
> sed -e 's/\(.\)\(...\)\(...\)\(...\)/\1\2 \3 \4 @@/' \
>     -e h -e 's/@@.*/@@/' -e 's/[^@]/& /g' -e G -e 's/\n//' \
>     -e 's/@@[^@]*@@//'
- r w -   r - -   r - -   2da.c
- r w -   r - -   r - -   3dalloc1.c
- r w -   r - -   r - -   3dalloc2.c
- r w -   r - -   r - -   3dalloc3.c
- r w -   r - -   r - -   3dalloc4.c
- r w -   r - -   r - -   4 @@ Testing
- r w -   r - -   r - -   @@ Testing
d r w x   r - x   r - x   Floppy
d r w x   r - x   r - x   IQ
d r w x   r - x   r - x   Primes
d r w x   r - x   r - x   SHA-256
$

What does the sed command do?

• Replace the permissions fields with a space after each group of 3 permissions bits and add @@ at the end of the permissions.
• Copy the modified line to the hold space.
• Replace the @@ and everything after it with just @@.
• Put a space after every non-@ character.
• Append the hold space to the pattern space with a newline.
• Remove the newline.
• Remove two @@ markers and everything between that isn't an @.

This leaves @@ markers in the file names untouched — witness the two names with spaces in them that contain @@.

For the record, the 11 lines passed to cut and then sed were:

-rw-r--r--   1 jleffler  staff     2362 Mar  6 19:48 2da.c
-rw-r--r--   1 jleffler  staff     1638 Mar  6 19:48 3dalloc1.c
-rw-r--r--   1 jleffler  staff     2870 Mar  6 19:48 3dalloc2.c
-rw-r--r--   1 jleffler  staff     2968 Mar  6 19:48 3dalloc3.c
-rw-r--r--   1 jleffler  staff     2096 Mar  6 19:48 3dalloc4.c
-rw-r--r--   1 jleffler  staff        0 Mar 21 16:46 4 @@ Testing
-rw-r--r--   1 jleffler  staff        0 Mar 21 16:47 @@ Testing
drwxr-xr-x   4 jleffler  staff      136 Mar  9 23:03 Floppy
drwxr-xr-x   8 jleffler  staff      272 Mar  6 19:48 IQ
drwxr-xr-x  33 jleffler  staff     1122 Mar  6 19:48 Primes
drwxr-xr-x  13 jleffler  staff      442 Mar  6 19:48 SHA-256

Answer 2

You simply don't. There's a whole universe of articles out there detailing why you should not be parsing ls output, but use combinations of tools like your shell's (most likely very comprehensive) file name globbing, find, and stat.

for example:

for name in * ;do echo $(stat -c '%A' "$name"):$name ; done

EDIT: stat gives you a lot of formats to help you achieve your desired output, and now that this gives you an unambiguous output (still, this can go wrong with things like newlines in file names), you can just use sed on the stat output in isolation. See man stat.