CODEWITHSUNDEEP
pythonpandascsv
mrvideo
mrvideo

Reputation: 39

How to import all csv files in one folder and make the filename the variable name in pandas?

I would like to automatically import all csv files that are in one folder as dataframes and set the dataframe's variable name to the respective filename.

For example, in the folder are the following three files: data1.csv, data2.csv and data3.csv

How can I automatically import all three files having three dataframes (data1, data2 and data3) as the result?

Upvotes: 1

Views: 1405

Answers (3)

OneTrickDragon
OneTrickDragon

Reputation: 87

Since the answer that was given includes an exec command, and munir.aygun already warned you what could go wrong with that approach. Now I want to show you the way to do it as Justin Ezequiel or munir.aygun already suggested:

import os
import glob
import pandas as pd

# Path to your data
path = r'D:\This\is\your\path'

# Get all .csv files at your path
allFiles = glob.glob(path + "/*.csv")

# Read in the data from files and safe to dictionary
dataStorage = {}
for filename in allFiles:
    name = os.path.basename(filename).split(".")[0]
    dataStorage[name] = pd.read_csv(filename)

# Can be used then like this (for printing here)
if "data1" in dataStorage:
    print(dataStorage["data1"])

Hope this can still be helpful.

Upvotes: 2

munir.aygun
munir.aygun

Reputation: 442

If you want to save dataframe as variable with own file name. But it is not secure. This could cause code injection.

import pandas
import os

path = "path_of_directory"
files = os.listdir(path) # Returns list of files in the folder which is specifed path

for file in files:
    if file.endswith(".csv"):# Checking wheter file endswith .csv
        # os.sep returns the separtor of operator system
        exec(f"{file[:-4]} = pandas.read_csv({path}+{os.sep}+{file})")

Upvotes: 2

Jon Clements
Jon Clements

Reputation: 142256

You can loop over the directory using pathlib and build a dictionary of name->DataFrame, eg:

import pathlib
import pandas as pd

dfs = {path.stem: pd.read_csv(path) for path in pathlib.Path('thepath/').glob(*.csv')}

Then access as dfs['test1'] etc...

Upvotes: 0

Related Questions