CODEWITHSUNDEEP
pythonpandas
denz franc
denz franc

Reputation: 105

How to Join Multiple DataFrames through loop in pandas?

df1: 

Id   Name
101  Rick
102  Nick
103  Jerry

df2:

Id   Class
101  A
102  B
103  C

df3:

Id   Grade
101  1
102  2
103  3

I need to merge Multiple Dataframes through a loop.

I tried:

data_frames= []
for file in os.listdir(folder_path):
    file.append(data_frames)

final_df = pd.merge(data_frames, how='left', on='Id')
TypeError: merge() missing 1 required positional argument: 'right'

Output:

    Id   Name  Class Grade
    101  Rick  A     1
    102  Nick  B     2
    103  Jerry C     3

Upvotes: 2

Views: 839

Answers (3)

Gilmar Vaz
Gilmar Vaz

Reputation: 102

#Import pandas
import pandas as pd


df1 = [[101, "Rick"],[102, "Nick"],[103, "Jerry"]]

df2 = [[101, "A"],[102, "B"],[103, "C"]]

df3 = [[101, 1],[102, 2],[103, 3]]

df1 = pd.DataFrame(df1, columns=["Id","Name"])
df2 = pd.DataFrame(df2, columns=["Id","Class"])
df3 = pd.DataFrame(df3, columns=["Id","Grade"])

df_final_mode1 = df1.merge(df2, left_on='Id', right_on='Id')
df_final_mode1 = df_final_mode1.merge(df3, left_on='Id', right_on='Id')

print('=== mode 1 - Merge ===')
print(df_final_mode1)

print()

print('=== mode 2 - Loop ===')
df_final_mode2 = df1

for index, (id, class_) in df2.iterrows():
    df_final_mode2.loc[df_final_mode2.Id == id, 'Class'] = df2.loc[df_final_mode2.Id == id, 'Class']

for index, (id, grade) in df3.iterrows():
    df_final_mode2.loc[df_final_mode2.Id == id, 'Grade'] = df3.loc[df_final_mode2.Id == id, 'Grade']

print(df_final_mode2)

enter image description here

Upvotes: 2

Cameron Riddell
Cameron Riddell

Reputation: 13407

You can use concat to merge a multiple dataframes on their index.

data_frames = [df1, df2, df3]

# if "Id" is not already the index on these dataframes
data_frames [d.set_index("Id") for d in data_frames]
out = pd.concat(data_frames, axis=1)

print(out)
      Name Class  Grade
Id
101   Rick     A      1
102   Nick     B      2
103  Jerry     C      3

Upvotes: 0

Quang Hoang
Quang Hoang

Reputation: 150745

On way is to use functools.reduce:

from functools import reduce

reduce(lambda x,y: x.merge(y,on='Id', how='left'), dataframes )

Output:

    Id   Name Class  Grade
0  101   Rick     A      1
1  102   Nick     B      2
2  103  Jerry     C      3

Upvotes: 1

Related Questions