CODEWITHSUNDEEP
pythonpandasimportpython-import
FlyUFalcon
FlyUFalcon

Reputation: 344

pandas: import multiple csv from subfolders if the name contains specific text

I have a folder located at C:\Users\Documents\folder and inside that folder there are 500 randomly named subfolders. Each subfolders has multiple csv files. I want to import csv files if only their name contain word client from those subfolders and concatenate the imported into one dataframe (lets hope I wont have any RAM issue).

Can someone help? Many Thanks.

Upvotes: 0

Views: 379

Answers (1)

Matthew Borish
Matthew Borish

Reputation: 3086

I think this should do it:

import os
import pandas as pd

source_dir = r'C:\Users\Documents\folder'

my_list = []

for root, dirnames, filenames in os.walk(source_dir):
    for f in filenames:

        if 'client' in f:

            my_list.append(pd.read_csv(os.path.join(root, f)))

concatted_df = pd.concat(my_list)

Upvotes: 2

Related Questions