CODEWITHSUNDEEP
python
Geo573
Geo573

Reputation: 142

Apply procedure to files in many subdirectories

I am trying to apply a procedure to thousands of files, but in many subdirectories.

I was thinking using os.listdir() first to list all subdirectories, than go look in each subdirectory and apply my procedure. My arborescence is as follow:

              subdir1 -> file, file, file, .....
              subdir2 -> file, file, file, .....
Directory  -> subdir3 -> file, file, file, .....
              subdir4 -> file, file, file, .....
              subdir5 -> file, file, file, .....

I can access the list of subdir with os.listdir() but not the files in the subdirectories, do you have an idea how to proceed ? Thanks

EDIT: When using MikeH method, in my case:

import os
from astropy.io import fits

ROOT_DIR='./'

for dirName, subdirList, fileList in os.walk(ROOT_DIR):
    for fname in fileList:
        hdul = fits.open(fname)

I get the error:

FileNotFoundError: [Errno 2] No such file or directory: 'lte08600-2.00+0.5.Alpha=+0.50.PHOENIX-ACES-AGSS-COND-2011-HiRes.fits'

And indeed if I try to check the path on the file, with print(os.path.abspath(fname) I can see that the path is wrong, it misses the subdirectories like /root/dir/fnam instead of root/dir/subdir/fname What is wrong in this ?

EDIT2: That's it I found out what was wrong, I have to join the path of the file, writing os.path.join(dirName,fname) instead of just fname each time. Thanks !

Upvotes: 1

Views: 238

Answers (1)

MikeH
MikeH

Reputation: 56

Something like this should work for you:

import os
ROOT_DIR='./'

for dirName, subdirList, fileList in os.walk(ROOT_DIR):
    for fname in fileList:
        # fully qualified file name is ROOT_DIR/dirname/fname
        performFunction(dirName, fname)

Upvotes: 1

Related Questions