CODEWITHSUNDEEP
pythonglobtext-parsing
MBasith
MBasith

Reputation: 1499

Python searching multiple directories and reading select files

I am looking for some help performing actions on a set of files in two different directories using Python.

I am attempting to:

  1. Search two different directories

  2. Find the 15 last modified files (comparing files in both directories)

  3. Read all 15 recently modified files line by line

I can accomplish reading through one file directory using glob. However, I cannot specify multiple directories. Is there another way I can accomplish this?

Below is my code which accomplishes grabbing the latest 15 files in dir1 but not dir2.

dir1 = glob.iglob("/dir1/data_log.*")
dir2 = glob.iglob("/dir2/message_log.*")

latest=heapq.nlargest(10, dir1, key=os.path.getmtime)
for fn in latest:
    with open(fn) as f:
        for line in f:
            print(line)

Upvotes: 0

Views: 1419

Answers (1)

Julien Spronck
Julien Spronck

Reputation: 15423

I'm not sure this is what you are after but if you were to use glob.glob instead of glob.iglob, you could do

dir1 = glob.glob("/dir1/data_log.*")
dir2 = glob.glob("/dir2/message_log.*")

latest=heapq.nlargest(10, dir1+dir2, key=os.path.getmtime)

And actually, if you don't like the idea of using lists (glob.glob) instead of generators (glob.iglob), you can do

from itertools import chain

dir1 = glob.iglob("/dir1/data_log.*")
dir2 = glob.iglob("/dir2/message_log.*")

latest=heapq.nlargest(10, chain(dir1, dir2), key=os.path.getmtime)

Upvotes: 4

Related Questions