CODEWITHSUNDEEP
pythonoperating-systemfilepathgloblistdir
alvas
alvas

Reputation: 122168

Ways to filter files in directory and join directory path - Python

Given a suffix and a directory path, I need to extract the full path of the files in the directory that ends with a given suffix.

Currently, I'm doing it as such:

import os
dir_path = '/path/to/dir'
suffix = '.xyz'
filenames = filter(lambda x: x.endswith(suffix), os.listdir(dir_path))
filenames = map(lambda x: os.path.join(dir_path, x), filenames)

I could also do it with glob:

import glob
dir_path = '/path/to/dir'
suffix = '.xyz'
glob.glob(dir_path+'*.'+suffix)

I understand that there's also pathlib that can check for suffixes using PurePath but I'm not sure what is the syntax for that.

Are there other ways of achieving the same filtered list of full paths to the files?

Upvotes: 0

Views: 4273

Answers (1)

msvalkon
msvalkon

Reputation: 12077

You can use a list comprehension to build the result in one go:

>>> [os.path.join(os.sep, x, dir_path) for x in os.listdir(dir_path) 
if x.endswith(suffix)]
['/home/msvalkon/foo.txt', 
 '/home/msvalkon/output.txt',
 '/home/msvalkon/remaining_warnings.txt',
 '/home/msvalkon/test.txt',
 '/home/msvalkon/hdr_chksum_failure.txt']

If dir_path is always an absolute path, you can use os.path.abspath(x) in place of the os.path.join().

For a large directory, it may be wise to use os.scandir which returns an iterator. This will be way faster.

>>> [entry.path for entry in os.scandir(dir_path) if entry.name.endswith(suffix)]
 ['/home/msvalkon/foo.txt', 
  '/home/msvalkon/output.txt',
  '/home/msvalkon/remaining_warnings.txt',
  '/home/msvalkon/test.txt',
  '/home/msvalkon/hdr_chksum_failure.txt']

Upvotes: 4

Related Questions