CODEWITHSUNDEEP
pythonpandascsvdataframe
callawayh
callawayh

Reputation: 91

Read and save multiple csv files from a for-loop

I am trying to read multiple csv files from a list of file paths and save them all as separate pandas dataframes.

I feel like there should be a way to do this, however I cannot find a succinct explanation. 

import pandas as pd

data_list = [['df_1','filepath1.csv'],
             ['df_2','filepath2.csv'],
             ['df_3','filepath3.csv']]

for name, filepath in data_list:
    name = pd.read_csv(filepath)

I have also tried:

data_list = [[df_1,'filepath1.csv'],[df_2,'filepath2.csv'],
             [df_3,'filepath3.csv']]

for name, filepath in data_list:
    name = pd.read_csv(filepath)

I would like to be able to call each dataframe by its assigned name.

Ex):

df_1.head()

Upvotes: 1

Views: 2650

Answers (3)

Sesquipedalism
Sesquipedalism

Reputation: 1733

df_dct = {name:pd.read_csv(filepath) for name, filepath in data_list}

would create a dictionary of DataFrames. This may help you organize your data.

You may also want to look into glob.glob to create your list of files. For example, to get all CSV files in a directory:

file_paths = glob.glob(my_file_dir+"/*.csv")

Upvotes: 1

Parthasarathy Subburaj
Parthasarathy Subburaj

Reputation: 4264

You can make sure of dictionary for this...

import pandas as pd
data_list = ['filepath1.csv', 'filepath2.csv', 'filepath3.csv']
d = {}
for _, i in enumerate(data_list):
    file_name = "df" + str(_)
    d[file_name] = pd.read_csv(filepath)

Here d is the dictionary which contains all your dataframes.

Upvotes: 0

Ali Hakim Taşkıran
Ali Hakim Taşkıran

Reputation: 109

I recommend you numpy. Read the csv files with numpy. 

from numpy import genfromtxt
my_data = genfromtxt('my_file.csv', delimiter=',')

You will get nd-array's. After that you can include them into pandas.

Upvotes: 0

Related Questions