CODEWITHSUNDEEP
pythonpandasfiledirectory
R Sem
R Sem

Reputation: 241

How to read each file from a folder and create seperate data frames for each file?

I am trying to get my code to read a folder containing various files. I was hoping to get Jupyter to read each file within that folder and create separate dataframes by taking the names of the files as the dataframe names.

So far I have the code:

import glob

path = r'C:\Users\SemR\Documents\Jupyter\Submissions' 
all_files = glob.glob(path + "/*.csv")

li = []

for filename in all_files:
    df = pd.read_csv(filename, index_col=None, header=0, usecols=['Date', 'Usage'])
    li.append(df)

frame = pd.concat(li, axis=0, ignore_index=True)

This code concatenates the data however I want separate data frames for each so that I can store the values separately. Is there something I can use instead?

Here are examples of how the CSV files look: enter image description here

These CSV files are in the same folder so I was hoping that when I run my code, new dataframes would be created with the same name as the CSV file name.

Thank you.

Upvotes: 2

Views: 5376

Answers (2)

Bibyutatsu
Bibyutatsu

Reputation: 296

From the question what I can suggest is that you have got different DataFrames stored in the list.

import glob

path = r'C:\Users\SemR\Documents\Jupyter\Submissions' 
all_files = glob.glob(path + "/*.csv")

li = []

for filename in all_files:
    df = pd.read_csv(filename, index_col=None, header=0, usecols=['Date', 'Usage'])
    li.append(df)

for dataframes in li:
    """ For getting the mean of a specific column """
    df.loc[:,"Usage"].mean()

Upvotes: 1

Martin Evans
Martin Evans

Reputation: 46759

A better approach to using different variables for each of your dataframes would be to load each dataframe into a dictionary.

The basename of each filename could be extracted using a combination of os.path.basename() and os.path.splitext().

For example:

d = {os.path.splitext(os.path.basename(f))[0] : pd.read_csv(f) for f in glob.glob('*test*.csv')}

Also, using *test* would avoid the need for the if in the comprehension.

Upvotes: 4

Related Questions