CODEWITHSUNDEEP
pythonpandaslist
Hao
Hao

Reputation: 13

Python pandas read multiple csv file and turn into dataframe

I tried to read multiple .csv files that have the same format from different folders. It turns out to be a list using .append and I tried to turn it into dataframe using .concat. But it didn't let me to do so. I also tried .os to read the data. It wouldn't work. Any suggestions?

test = []
train = []
for f in testdata:
    test.append(pd.read_csv(f, skiprows = 5, sep = ',', names = 
         ['time','avg_rss12','var_rss12','avg_rss13','var_rss13','avg_rss23','var_rss23']))
for f in traindata:
    train.append(pd.read_csv(f, skiprows = 5, sep = ',', names = 
         ['time','avg_rss12','var_rss12','avg_rss13','var_rss13','avg_rss23','var_rss23']))

here is one .csv file data

Upvotes: 0

Views: 270

Answers (1)

gallen
gallen

Reputation: 1292

You mention attempting pandas concat, but don't specify how it failed. It is the way you would want to go. Using just your test loop from the question:

import pandas as pd


frames = []
for f in testdata:
    df = pd.read_csv(f, skiprows=5, sep=',', names=['time','avg_rss12','var_rss12','avg_rss13','var_rss13','avg_rss23','var_rss23'])
    frames.append(df)
combined = pd.concat(frames)

What gets passed into concat() needs to be a list of DataFrame instances, which it will combine into a single frame.

Upvotes: 1

Related Questions