CODEWITHSUNDEEP
pythonfilesystems
Saif Ahmed
Saif Ahmed

Reputation: 1

Iterate over multiple folders at once

I am looking to loop over my data folders categorized by year eg. enter image description here

Example file name: EastSales_20200101_house.txt

Rather than hardcoding through each year folder path one by one, what would be an alternative for looping through the year folders and categorize the data in the year folders by month.

enter image description here

from pathlib import Path
from datetime import datetime
import os

my_data = Path("/Users/me/Documents/my_data")
#check if file exists
for file in my_data.iterdir():
  if file.is_file():
    directory = file.parent
    extension = file.suffix
    old_name  = file.stem
    region,old_date,categ = old_name.split('_')
     #rename file
    old_date = datetime.strptime(old_date,"%Y%m%d")
    date = datetime.strftime(old_date, "%Y%m%d")
    new_date = f'{date}{extension}'

    #categorize by year
    year = datetime.strftime(old_date, "%Y")
    new_path = my_data.joinpath(year)
    
    #check if folder exists
    if not new_path.exists():
      new_path.mkdir()

    #create new path in directory
    new_file_path = new_path.joinpath(newfile)
    file.replace(new_file_path)

# Currently looping over months by hardcoding path
my_data_2020 = Path("/Users/me/Documents/my_data/2020")
#check if file exists
for file in my_data_2020.iterdir():
  if file.is_file():
    directory_2020 = file.parent
    extension = file.suffix
    old_name_2020  = file.stem

     #rename file
    old_date_2020 = datetime.strptime(old_date_2020,"%Y%m%d")
    date_2020 = datetime.strftime(old_date_2020, "%Y%m%d")
    new_date_2020 = f'{date_2020}{extension}'

    #categorize by month
    month = datetime.strftime (old_date_2020, "%m)
    new_path_2020 = my_data.joinpath(year)
    
    #check if folder exists
    if not new_path_2020.exists():
      new_path_2020.mkdir()

    #create new path in directory
    new_file_path_2020 = new_path_2020.joinpath(newfile_2020)
    file.replace(new_file_path_2020)

Upvotes: 0

Views: 647

Answers (1)

Matte
Matte

Reputation: 114

The actual walk through the directories works as you have coded it. If you replace the contents of the inner loop with a simple print statement you can see that each file is found:

import os
rootdir = 'C:/Users/sid/Desktop/test'
    
for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        print(os.path.join(subdir, file))

Upvotes: 2

Related Questions