CODEWITHSUNDEEP
bashfile
Paul Draper
Paul Draper

Reputation: 83255

Filter list of files to those that exist

How do I filter a list of files to the ones that exist?

For example,

echo 'a.txt
does/not.exist
b.txt' | <???>

would print

a.txt
b.txt

Upvotes: 25

Views: 9082

Answers (6)

Thiago Barcala
Thiago Barcala

Reputation: 7333

It is possible to pass multiple files to stat, and if the file doesn't exist it will just print a message to STDERR. So you can do this:

<input> | xargs stat --printf '%n\n' 2> /dev/null

Or if you are using null-terminated paths (which I recommend and is not possible with ls):

<input> | xargs -0 stat --printf '%n\0' 2> /dev/null

Upvotes: 2

mklement0
mklement0

Reputation: 437773

If you have GNU xargs, use -d '\n' to ensure that filenames (including directories) with embedded spaces are handled correctly, by splitting the input into whole lines rather than also by intra-line whitespace.

echo 'a.txt
does/not.exist
b.txt' | xargs -d '\n' ls -1df 2>/dev/null

Note:

  • The ls command emits an error for each non-existent input path, which 2>/dev/null ignores, while echoing existing paths as-is.

  • Option -1 prints each path on its own line, -d prevents recursing into directories, and -f prevents sorting of the input paths (if you actually want sorting, omit the f).

On macOS/BSD, xargs doesn't support -d, which requires a workaround via NUL-separated input using tr and xargs's -0 option:

echo 'a.txt
does/not.exist
b.txt' | tr '\n' '\0' | xargs -0 ls -1df 2>/dev/null

Upvotes: 7

Tommy Jollyboat
Tommy Jollyboat

Reputation: 239

As a one-liner, and pure bash for speed (improved from mklement0's answer, would have commented if I'd had the rep):

{ ls; echo does/not.exist; } | while IFS= read -r f; do [[ -f "$f" ]] && echo "$f"; done

Upvotes: 5

rjmunro
rjmunro

Reputation: 28056

I would use bash's if to check for the files. It ends up a bit less compact, but I think it's clearer to read, and easier to do something with each found file in the results.

It is compatible with filenames with spaces.

echo 'a.txt
does/not.exist
b.txt' | while read filename
do
  if [[ -f "$filename" ]]
  then
    echo $filename # Or do something else with the files here
  fi
done

Upvotes: 1

fedorqui
fedorqui

Reputation: 289725

You can ls -d the files and see which ones get some output. Since you have those in a string, just pipe the list and use xargs to be able to ls them.

To hide errors, redirect those to /dev/null. All together, xargs ls -d 2>/dev/null makes it:

$ echo 'a.txt
b.txt
other' | xargs ls -d 2>/dev/null
a.txt
b.txt

As you see, xargs ls -d executes ls -d to all the arguments given. 2>/dev/null gets rid of the stderr messages.

Upvotes: 25

ThorSummoner
ThorSummoner

Reputation: 18109

First thing I came up with, use stats's exit code in a while read loop:

<input> | while IFS= read -r f; do stat "$f" &>/dev/null && echo "$f"; done

Note that this solution is slow, because it loops in shell code, and calls an external utility (creates a child process, stat) in every iteration.

Upvotes: 1

Related Questions