CODEWITHSUNDEEP
pythonpandascsv
BeeOnRope
BeeOnRope

Reputation: 64925

Renaming columns based on csv filename in pandas

Given that I'm reading N csv files and merging them into a single Pandas DataFrame like:

dfs = [pd.read_csv(f) for f in list_of_files]
df = pd.concat(dfs, axis=1)

How can I rename the columns from each file, so that they include a suffix based on the file name?

For example, if files f1 and f2 have the following contents:

f1:

A
1
2
3

f2:

B
4
5
6

Then the column-wise concat above produces:

A  B
1  4
2  5
3  6

... but I want:

A_f1  B_f2
   1     4
   2     5
   3     6

Upvotes: 4

Views: 1844

Answers (3)

Erfan
Erfan

Reputation: 42916

You can add suffixes to your df's before you use pd.concat:

lst_dfs = []

for file in list_of_files:
    df = pd.read_csv(file)
    df = df.add_suffix(f'_{file}')
    lst_dfs.append(df)

df_all = pd.concat(lst_dfs, axis=1)

Edit

A small test with two csv files

list_of_files = ['table1.csv', 'table2.csv']

lst_dfs = []

for file in list_of_files:
    df = pd.read_csv(file, sep='|')
    df = df.add_suffix(f'_{file}')
    lst_dfs.append(df)

df_all = pd.concat(lst_dfs, axis=1)

#Optional to remove the filename extension
df_all.columns = df_all.columns.str.replace('.csv', '')

print(df_all)
  key_table1  value_table1 key_table2  value_table2
0          A     -0.323896          B      0.050969
1          B      0.073764          D     -0.228590
2          C     -0.798652          E     -2.160319
3          D      0.970627          F     -0.213936

Upvotes: 3

Ben.T
Ben.T

Reputation: 29635

you can use add_suffix such as:

dfs = [pd.read_csv(f).add_suffix('-' + str(f)) for f in list_of_files]

Upvotes: 3

BENY
BENY

Reputation: 323236

Change your dfs to dict 

dfs = {'f'+str(i+1) : pd.read_csv(f) for i,f in enumerate(list_of_files)}

Then using cancat 

s=pd.concat(dfs,1)
s.columns=s.columns.map('{0[1]}_{0[0]}'.format) 
s
Out[311]: 
   A_f1  B_f2
0     1     4
1     2     5
2     3     6

Upvotes: 3

Related Questions