CODEWITHSUNDEEP
pythonpython-3.xpandas
asd
asd

Reputation: 1309

Loop over common files in multiple folders

How could I find files with the same filename in multiple folders, and then perform the same operations?

def findCommonDeep(path1, path2):
    return set.intersection(*(set(os.path.relpath(os.path.join(root, file), path) for root, _, files in os.walk(path) for file in files) for path in (path1, path2)))

lista = []
for x in [2010, 2017, 2020]:
    if x > 2015:
        filepath = rf'My Documents\Analysis\{x}\LateAnalysis\*.csv'
    else:
        filepath = rf'My Documents\Analysis\{x}\Early_Analysis\*.csv'

    fname = os.path.basename(filepath, filepath)
    findCommonDeep(fname)

    for file in glob.glob(filename_common):
        df = pd.read_csv(file)
        df = df.set_index('date')
        lista.append(df)

Upvotes: 1

Views: 240

Answers (1)

Umar.H
Umar.H

Reputation: 23099

As you've tagged pandas, let's use pandas and pathlib to return a dictionary of files with similar names:

from pathlib import Path 
import pandas as pd 

def return_similair_files(start_dir : str) -> dict:

   all_files = Path(start_dir).rglob('*.csv')
   df = pd.DataFrame({'files' : all_files})
   df['name'] = df['files'].apply(lambda x : x.name) #pathlib method. 

   return df.groupby('name')['files'].agg(list).to_dict()

This will return a dictionary of files, like so:

{'file_1.csv' : [list_of_paths]}

You can then operate on them as you need.

Upvotes: 1

Related Questions