How to get the list of files in a directory in a shell script?

Question

I'm trying to get the contents of a directory using shell script.

My script is:

for entry in `ls $search_dir`; do
    echo $entry
done

where $search_dir is a relative path. However, $search_dir contains many files with whitespaces in their names. In that case, this script does not run as expected.

I know I could use for entry in *, but that would only work for my current directory.

I know I can change to that directory, use for entry in * then change back, but my particular situation prevents me from doing that.

I have two relative paths $search_dir and $work_dir, and I have to work on both simultaneously, reading them creating/deleting files in them etc.

So what do I do now?

PS: I use bash.

Answer 1

If you have installed the tree tool or can install it, it's a pretty good tool. It can list all the files in the specified dir and its subdir. Also can filter the output in many ways:

-a            All files are listed.
-d            List directories only.
-l            Follow symbolic links like directories.
-f            Print the full path prefix for each file.
-x            Stay on current filesystem only.
-L level      Descend only level directories deep.
-R            Rerun tree when max dir level reached.
-P pattern    List only those files that match the pattern given.
-I pattern    Do not list files that match the given pattern.

etc. Just tree --help for more info.

Answer 2

Get list of files in a directory in shell and in bash, 4 tips

Some remarks

• Seaching for files is not searching for entries, under Un*x, Everything is a file, so under a path, you could find sockets, devices, fifos, and lot of other things which are not files.
• Correct accepted answer from Ignacio Vazquez-Abrams will fail if $search_dir is empty or if $search_dir doesn't exist.
• Under bash, you could use arrays,
• and use bash's globstar for recursivity instead of find.

Posix shell first:

If you

search_dir="/the/path to/base/dir"
for entry in "$search_dir"/* ; do

(note the use of quote, for keeping space in path name)

From there, script execution will continue even if

• there's nothing in directory
• the directory doesn't exist (wrong path)
• things presents in directory are not files (named fifo, unix socket, audio device, hard disk, ...).

So next step have to be:

    if [ -f "$entry" ] ;then
        printf 'Now, you could quietly do something with "%s".\n' "$entry"
        # ...
    fi
done

Now bash.

bash array

I often use this:

entries=("$search_dir"/*)

But, same remark: this will produce an array containing ("/the/path to/base/dir/*") in case of no file or wrong path.

You could clean 1st wrong field by using

[[ ${entries[0]} == "$search_dir/*" ]] && unset entries[0]

Or you could do a quick loop over the array, see at bottom of this answer...

Then

for entry in "${entries[@]}";do
    if [[ -f $entry ]] ;then  # ensuring entry is a file.
        'Now, you could quietly do something with "%s".\n' "$entry"
        # ...
    fi
done

With bash array, you could show the content of your array with full path, by:

printf ' - %s\n' "${entries[@]}"

And, for showing only file names:

printf ' - %s\n' "${entries[@]#$search_dir/}"

Recursion using globstar:

From man bash:

• globstar
  If set, the pattern ** used in a pathname expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a /, only directories and subdirectories match.

globstar is off by default:

shopt globstar 
globstar        off
shopt -s globstar 

Then

entries=("$search_dir"/**)
printf ' - %s\n' "${entries[@]#$search_dir/}"

will print every entries under $search_dir/.

Searching for files.

Quick loop over array for dropping entries which are not files.

for i in ${!entries[@]};do
    [[ -f ${entries[i]} ]] || unset entries[i]
done

Then

printf ' - %s\n' "${entries[@]#$search_dir/}"

will print every files under $search_dir/.

Building array while searching for files

If your $search_dir do hold a lot of non-file entries, you'd better build array from tested entries:

for entry in "$search_dir"/**;do
    entries+=("$entry")
done

printf ' - %s\n' "${entries[@]#$search_dir/}"

Answer 3

There is simpler answer with using mapfile!

mapfile -t file_list < <(ls "${some_path}")

Then you can use file_list list with all contents in some_path! If you want just file list, use find instead of ls

Ex)

// check count.
echo "${#file_list[@]}"

// check list.
for file_name in "${file_list[@]}"; do
  echo "$file_name"
done

Answer 4

search_dir=/the/path/to/base/dir
for entry in "$search_dir"/*
do
  echo "$entry"
done

Answer 5

The accepted answer will not return files prefix with a '.' To do that use

for entry in "$search_dir"/* "$search_dir"/.[!.]* "$search_dir"/..?*
do
  echo "$entry"
done

Answer 6

Similar to Accepted answer - but lists only file names instead of full paths:

This seems to have been answered a while ago, but I guess I want to also contribute an answer that just lists the files in the desired directory, as opposed to the full paths.

    #search_dir=/the/path/to/base/dir/
    IFS=$'\n' #for in $() splits based on IFS
    search_dir="$(pwd)"
    for entry in $(ls $search_dir)
    do
        echo $entry
    done

If you also wanted to filter for a specific file you would add a grep -q statement.

    #search_dir=/the/path/to/base/dir/
    IFS=$'\n' #for in $() splits based on IFS
    search_dir="$(pwd)"
    for entry in $(ls $search_dir)
    do
        if grep -q "File should contain this entire string" <<< $entry; then
        echo "$entry"
        fi
    done

References:

More information about IFS can be found here.

More information about finding substrings in shell can be found here.

Answer 7

How to get the list of files in a directory in a shell script?

In addition to the most-upvoted answer by @Ignacio Vazquez-Abrams, consider the following solutions which also all work, depending on what you are trying to do. Note that you can replace "path/to/some/dir" with . in order to search in the current directory.

1. List different types of files using find and ls

References:

1. For find, see this answer. See also my comment here.
2. For ls, see linuxhandbook.com: How to List Only Directories in Linux

Tip: for any of the find examples below, you can pipe the output to sort -V if you'd like it sorted.

Example:

find . -maxdepth 1 -type f | sort -V

List only regular files (-type f) 1 level deep:

# General form
find "path/to/some/dir" -maxdepth 1 -type f

# In current directory
find . -maxdepth 1 -type f

List only symbolic links (-type l) 1 level deep:

# General form
find "path/to/some/dir" -maxdepth 1 -type l

# In current directory
find . -maxdepth 1 -type l

List only directories (-type d) 1 level deep:

Note that for the find example here, we also add -mindepth 1 in order to exclude the current directory, ., which would be printed as . at the top of the directory list otherwise. See here: How to exclude this / current / dot folder from find "type d"

# General form
find "path/to/some/dir" -mindepth 1 -maxdepth 1 -type d

# In current directory
find . -mindepth 1 -maxdepth 1 -type d

# OR, using `ls`:
ls -d

Combine some of the above: list only regular files and symbolic links (-type f,l) 1 level deep:

Use a comma (,) to separate arguments to -type:

# General form
find "path/to/some/dir" -maxdepth 1 -type f,l

# In current directory
find . -maxdepth 1 -type f,l

2. Capture the output of any command into a bash indexed array, with elements separated by the newline char (\n)

However, $search_dir contains many files with whitespaces in their names. In that case, this script does not run as expected.

This is solved by telling bash to separate elements in the string based on the newline char \n instead of the space char--which is the default IFS (Internal Field Separator--see The Meaning of IFS in Bash Scripting) variable used by bash. To do this, I recommend using the mapfile command.

The bash script static code analyzer tool named shellscript recommends using mapfile or read -r whenever you want to read in a string into a bash array, separating elements based on the newline char (\n). See: https://github.com/koalaman/shellcheck/wiki/SC2206.

Update: to see examples of how to do this with both mapfile and read -r see my answer here: How to read a multi-line string into a regular bash "indexed" array. I now prefer to use read -r instead of mapfile, because mapfile will KEEP any empty lines as elements in the array, if any exist, which I do NOT want, whereas read -r [again, my preference now] will NOT keep empty lines as elements in the array.

(Back to my original answer:)

Here is how to convert a newline-separated string into a regular bash "indexed" array with the mapfile command.

# Capture the output of `ls -1` into a regular bash "indexed" array.
# - includes both files AND directories!
mapfile -t allfilenames_array <<< "$(ls -1)"
# Capture the output of `find` into a regular bash "indexed" array
# - includes directories ONLY!
# Note: for other `-type` options, see `man find`.
mapfile -t dirnames_array \
    <<< "$(find . -mindepth 1 -maxdepth 1 -type d | sort -V)"

Notes:

1. We use ls -1 (that's a "dash numeral_one") in order to put each filename on its own line, thereby separating them all by the newline \n char.
2. If you'd like to Google it, <<< is called a "here string" in bash.
3. See mapfile --help, or help mapfile, for help.

Full code example:

From file array_list_all_files_and_directories.sh in my eRCaGuy_hello_world repo:

echo "Output of 'ls -1'"
echo "-----------------"
ls -1
echo ""

# Capture the output of `ls -1` into a regular bash "indexed" array.
# - includes both files AND directories!
mapfile -t allfilenames_array <<< "$(ls -1)"
# Capture the output of `find` into a regular bash "indexed" array
# - includes directories ONLY!
# Note: for other `-type` options, see `man find` and see my answer here:
# https://stackoverflow.com/a/71345102/4561887
mapfile -t dirnames_array \
    <<< "$(find . -mindepth 1 -maxdepth 1 -type d | sort -V)"

# Get the number of elements in each array
allfilenames_array_len="${#allfilenames_array[@]}"
dirnames_array_len="${#dirnames_array[@]}"

# 1. Now manually print all elements in each array

echo "All filenames (files AND dirs) (count = $allfilenames_array_len):"
for filename in "${allfilenames_array[@]}"; do
    echo "    $filename"
done
echo "Dirnames ONLY (count = $dirnames_array_len):"
for dirname in "${dirnames_array[@]}"; do
    # remove the `./` from the beginning of each dirname
    dirname="$(basename "$dirname")"
    echo "    $dirname"
done
echo ""

# OR, 2. manually print the index number followed by all elements in the array

echo "All filenames (files AND dirs) (count = $allfilenames_array_len):"
for i in "${!allfilenames_array[@]}"; do
    printf "  %3i: %s\n" "$i" "${allfilenames_array["$i"]}"
done
echo "Dirnames ONLY (count = $dirnames_array_len):"
for i in "${!dirnames_array[@]}"; do
    # remove the `./` from the beginning of each dirname
    dirname="$(basename "${dirnames_array["$i"]}")"
    printf "  %3i: %s\n" "$i" "$dirname"
done
echo ""

Here is the example output of the code block just above being run inside the eRCaGuy_hello_world/python dir of my eRCaGuy_hello_world repo:

eRCaGuy_hello_world/python$ ../bash/array_list_all_files_and_directories.sh
Output of 'ls -1'
-----------------
autogenerate_c_or_cpp_code.py
autogenerated
auto_white_balance_img.py
enum_practice.py
raw_bytes_practice.py
slots_practice
socket_talk_to_ethernet_device.py
textwrap_practice_1.py
yaml_import

All filenames (files AND dirs) (count = 9):
    autogenerate_c_or_cpp_code.py
    autogenerated
    auto_white_balance_img.py
    enum_practice.py
    raw_bytes_practice.py
    slots_practice
    socket_talk_to_ethernet_device.py
    textwrap_practice_1.py
    yaml_import
Dirnames ONLY (count = 3):
    autogenerated
    slots_practice
    yaml_import

All filenames (files AND dirs) (count = 9):
    0: autogenerate_c_or_cpp_code.py
    1: autogenerated
    2: auto_white_balance_img.py
    3: enum_practice.py
    4: raw_bytes_practice.py
    5: slots_practice
    6: socket_talk_to_ethernet_device.py
    7: textwrap_practice_1.py
    8: yaml_import
Dirnames ONLY (count = 3):
    0: autogenerated
    1: slots_practice
    2: yaml_import

Answer 8

ls $search_path ./* |grep ".txt"|
while IFS= read -r line
do 
   echo "$line"
done

Answer 9

$ pwd; ls -l
/home/victoria/test
total 12
-rw-r--r-- 1 victoria victoria    0 Apr 23 11:31  a
-rw-r--r-- 1 victoria victoria    0 Apr 23 11:31  b
-rw-r--r-- 1 victoria victoria    0 Apr 23 11:31  c
-rw-r--r-- 1 victoria victoria    0 Apr 23 11:32 'c d'
-rw-r--r-- 1 victoria victoria    0 Apr 23 11:31  d
drwxr-xr-x 2 victoria victoria 4096 Apr 23 11:32  dir_a
drwxr-xr-x 2 victoria victoria 4096 Apr 23 11:32  dir_b
-rw-r--r-- 1 victoria victoria    0 Apr 23 11:32 'e; f'

$ find . -type f
./c
./b
./a
./d
./c d
./e; f

$ find . -type f | sed 's/^\.\///g' | sort
a
b
c
c d
d
e; f

$ find . -type f | sed 's/^\.\///g' | sort > tmp

$ cat tmp
a
b
c
c d
d
e; f

---

Variations

$ pwd
/home/victoria

$ find $(pwd) -maxdepth 1 -type f -not -path '*/\.*' | sort
/home/victoria/new
/home/victoria/new1
/home/victoria/new2
/home/victoria/new3
/home/victoria/new3.md
/home/victoria/new.md
/home/victoria/package.json
/home/victoria/Untitled Document 1
/home/victoria/Untitled Document 2

$ find . -maxdepth 1 -type f -not -path '*/\.*' | sed 's/^\.\///g' | sort
new
new1
new2
new3
new3.md
new.md
package.json
Untitled Document 1
Untitled Document 2

Notes:

• . : current folder
• remove -maxdepth 1 to search recursively
• -type f : find files, not directories (d)
• -not -path '*/\.*' : do not return .hidden_files
• sed 's/^\.\///g' : remove the prepended ./ from the result list

Answer 10

Here's another way of listing files inside a directory (using a different tool, not as efficient as some of the other answers).

cd "search_dir"
for [ z in `echo *` ]; do
    echo "$z"
done

echo * Outputs all files of the current directory. The for loop iterates over each file name and prints to stdout.

Additionally, If looking for directories inside the directory then place this inside the for loop:

if [ test -d $z ]; then
    echo "$z is a directory"
fi

test -d checks if the file is a directory.

Answer 11

This is a way to do it where the syntax is simpler for me to understand:

yourfilenames=`ls ./*.txt`
for eachfile in $yourfilenames
do
   echo $eachfile
done

./ is the current working directory but could be replaced with any path
*.txt returns anything.txt
You can check what will be listed easily by typing the ls command straight into the terminal.

Basically, you create a variable yourfilenames containing everything the list command returns as a separate element, and then you loop through it. The loop creates a temporary variable eachfile that contains a single element of the variable it's looping through, in this case a filename. This isn't necessarily better than the other answers, but I find it intuitive because I'm already familiar with the ls command and the for loop syntax.

Answer 12

find "${search_dir}" "${work_dir}" -mindepth 1 -maxdepth 1 -type f -print0 | xargs -0 -I {} echo "{}"

Answer 13

The other answers on here are great and answer your question, but this is the top google result for "bash get list of files in directory", (which I was looking for to save a list of files) so I thought I would post an answer to that problem:

ls $search_path > filename.txt

If you want only a certain type (e.g. any .txt files):

ls $search_path | grep *.txt > filename.txt

Note that $search_path is optional; ls > filename.txt will do the current directory.

Answer 14

for entry in "$search_dir"/* "$work_dir"/*
do
  if [ -f "$entry" ];then
    echo "$entry"
  fi
done