CODEWITHSUNDEEP
bashshellloopscsvrecursion
Nicholas Hazel
Nicholas Hazel

Reputation: 3750

Recursively list CSV of filename,dir

Pretend my root was: foo

I have a folder structure of:

-foo
  -123
    -file.jpg
    -file2.jpg
  -456
    -file.jpg
    -file2.jpg

The directory could potentially be a few folders deeper.

If I had the root as such:

ROOT=foo

How would I go about listing every filename and their path in a CSV sort of format in a shell script?

I tried a few of the answers but I'm struggling to make it output a long string of CSV.

I'd expect something like:

file.jpg,123,file2.jpg,123,file.jpg,456,file2.jpg,456

So, I tried this:

# make a var for files in current folder
arr=(./*)
CSVRETURN=""
# simple loop to iterate files
for ((i=0; i<${#arr[@]}; i++)); do
    #do something to each element of array
    CSVRETURN="$CSVRETURN","${arr[$i]}"
done
echo $CSVRETURN

It keeps adding periods to my output and I don't believe I even have the recursive portion correct. Could somebody please point me in the right direction or a doc that may help?

I've done very little with shell scripts but I'm trying :) Basically going to take this answer and export to a CSV for an entire directory and provide links for images with their dir path (relative to initial dir) for easy filtering to import to Excel.

Upvotes: 0

Views: 1486

Answers (2)

etopylight
etopylight

Reputation: 1319

Instead of bash, this can be easily done if you have GNU find

find foo -type f -printf '"%f","%h"\n' | paste -d, -s -
  • %f for filename
  • %h for leading directory path
  • paste -d, -s - concatenates newline separated result into a comma separated single line

Or a more general solution with BSD find since it does not support the -printf option

find foo -type f -print | sed 's|^\(.*\)/\(.*\)|"\2","\1"|' | paste -d, -s -

Output

"file1.jpg","foo/123","file2.jpg","foo/123","file1.jpg","foo/456","file2.jpg","foo/456"

To generate a quick test case, you can use

mkdir -p foo/{123,456}
touch foo/{123,456}/file{1..2}.jpg

to produce a directory structure like

foo/
├── 123
│   ├── file1.jpg
│   └── file2.jpg
└── 456
    ├── file1.jpg
    └── file2.jpg

Upvotes: 1

hmedia1
hmedia1

Reputation: 6180

Since you want:

  • No leading period
  • To handle folders
  • Single line CSV

You could have two conventions (based on my understanding of your example):

  1. List like: 
    full/folder/file  ,  full/folder/file
  2. List like: 
    full/folder  ,  file  ,  full/folder  ,  file

Assuming my current directory foo looks like this:

14:37:14 ツ :foo >ls -R
sublevel1        456            123

./sublevel1:
123

./sublevel1/123:
file2.jpg file.jpg

./456:
file2.jpg file.jpg

./123:
file2.jpg file.jpg

Then for the first convention:

#!/usr/bin/env bash
first=1
find . -type f -print0 | while IFS= read -rd '' file ; do 
    if [ "$first" == "1" ]; then 
        filestr="\"$(cut -f2- -d'/' <<< "${file}")\""
        unset first
   else 
        filestr=",\"$(cut -f2- -d'/' <<< "${file}")\""  
        fi
   printf "%s" "$filestr"
done

This would give you the following result:

"sublevel1/123/file.jpg","sublevel1/123/file2.jpg","456/file.jpg","456/file2.jpg","123/file.jpg","123/file2.jpg"

If you changed the -print0 to -printf "%h\0%f\0", then you'd get an output like the second convention above:

"sublevel1/123","file.jpg","sublevel1/123","file2.jpg","456","file.jpg","456","file2.jpg","123","file.jpg","123","file2.jpg"

Notes:

  • The null (\0 , -print0) handling makes this a bit more reliable to handle strange file names, maybe even ones with embedded newlines.
  • The printf "%s" format is fairly resilient in it's handling of names also when the string is passed to the program properly
  • The if block is just to ensure a comma is not prefixed to the very first file
  • The inefficiency of a "while" loop shouldn't be noticeable for any conceivable use you could have for wanting a single string csv list like this.
  • I added sublevel1 to demonstrate multiple folder levels.

For more robust applications, consider serialized JSON, or some other data representation format where processing of lists can be handled in a more complete manner.

Upvotes: 0

Related Questions