CODEWITHSUNDEEP
pythondirectory
Wolf
Wolf

Reputation: 3209

Iterating through directories with Python

I need to iterate through the subdirectories of a given directory and search for files. If I get a file I have to open it and change the content and replace it with my own lines.

I tried this:

import os

rootdir ='C:/Users/sid/Desktop/test'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        f=open(file,'r')
        lines=f.readlines()
        f.close()
        f=open(file,'w')
        for line in lines:
            newline = "No you are not"
            f.write(newline)
        f.close()

but I am getting an error. What am I doing wrong?

Upvotes: 252

Views: 509005

Answers (3)

ChrisProsser
ChrisProsser

Reputation: 13108

The actual walk through the directories works as you have coded it. If you replace the contents of the inner loop with a simple print statement you can see that each file is found:

import os
rootdir = 'C:/Users/sid/Desktop/test'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        print(os.path.join(subdir, file))

Upvotes: 473

Pedro Lobito
Pedro Lobito

Reputation: 99011

From python >= 3.5 onward, you can use **, glob.iglob(path/**, recursive=True) and it seems the most pythonic solution, i.e.:

import glob, os

for filename in glob.iglob('/pardadox-music/**', recursive=True):
    if os.path.isfile(filename): # filter dirs
        print(filename)

Output:

/pardadox-music/modules/her1.mod
/pardadox-music/modules/her2.mod
...

Notes:

  1. glob.iglob

    glob.iglob(pathname, recursive=False)

    Return an iterator which yields the same values as glob() without actually storing them all simultaneously.

  2. If recursive is True, the pattern '**' will match any files and zero or more directories and subdirectories.

  3. If the directory contains files starting with . they won’t be matched by default. For example, consider a directory containing card.gif and .card.gif:

    >>> import glob
>>> glob.glob('*.gif') ['card.gif'] 
>>> glob.glob('.c*')['.card.gif']

  4. You can also use rglob(pattern), which is the same as calling glob() with **/ added in front of the given relative pattern.

Upvotes: 28

joelostblom
joelostblom

Reputation: 49064

Another way of returning all files in subdirectories is to use the pathlib module, introduced in Python 3.4, which provides an object oriented approach to handling filesystem paths (Pathlib is also available on Python 2.7 via the pathlib2 module on PyPi):

from pathlib import Path

rootdir = Path('C:/Users/sid/Desktop/test')
# Return a list of regular files only, not directories
file_list = [f for f in rootdir.glob('**/*') if f.is_file()]

# For absolute paths instead of relative the current dir
file_list = [f for f in rootdir.resolve().glob('**/*') if f.is_file()]

Since Python 3.5, the glob module also supports recursive file finding:

import os
from glob import iglob

rootdir_glob = 'C:/Users/sid/Desktop/test/**/*' # Note the added asterisks
# This will return absolute paths
file_list = [f for f in iglob(rootdir_glob, recursive=True) if os.path.isfile(f)]

The file_list from either of the above approaches can be iterated over without the need for a nested loop:

for f in file_list:
    print(f) # Replace with desired operations

Upvotes: 41

Related Questions