CODEWITHSUNDEEP
pythoncsvpandas
Shubham R
Shubham R

Reputation: 7644

Join multiple csv files from a folder into a single csv python

i have around 100 csv files in a folder.

/path/to/directory/*.csv
 it has files abc.csv,dsf.csv,rgfb.csv.....etc

a view of csv file.

182 a   1   4   242 52450
182 a   1   2   242 7176
182 c   1   1   242 7176
182 c   1   1   242 7410

i want to take all this csv from the directory and make it in one csv. There are no column names, but all the csv has same no of columns(i.e 5), and i want to join all csv, and put it in pandas dataframe and give column names as 

col1  col2  col3  col4  col5
data  data  data  data  data
...   ...    ...   ...   ...

what i tried was.

import os
csv_list = []
for root, dirs,files in os.walk("path/to/directory", topdown=True):
for name in files:
    csv_list.append(os.path.join(root, name))

i got the list of csv

then i did 

import pandas as pd
combined_csv = pd.append( [ pd.read_csv(f) for f in csv_list ] )

but it is appending horizontally and not vertically.

Also i have to give column names to 'combined_csv' is there any better way?

Upvotes: 1

Views: 3404

Answers (1)

jezrael
jezrael

Reputation: 862511

I think you need concat with parameter axis=1 if need append vertically:

combined_csv = pd.concat([ pd.read_csv(f, header=None) for f in csv_list ], axis=1)

And if need append horizontally is default parameter axis=0 which can be omit:

import pandas as pd
combined_csv = pd.concat([ pd.read_csv(f, header=None) for f in csv_list ], ignore_index=True)

If need set column names use parameter names:

names = ['col1','col2','col3','col4','col5']
combined_csv = pd.concat([ pd.read_csv(f, header=None, names = names) for f in csv_list ],
                           ignore_index=True)

Upvotes: 3

Related Questions