CODEWITHSUNDEEP
pythonpython-3.xpandasnumpyconcatenation
MTALY
MTALY

Reputation: 1772

Pandas concat with outer join - Reindexing only valid with uniquely valued Index objects

I have 6 dataframes that contain information about unique customers like one df for emails the second one for first name .. etc

I am doing concat with outer join to have one df with all customer and information columns.

This is what I did so far:

info_dfs = (df1,df2,df3,df4,df5,df6)

Now the concatenation:

all_merged = pd.concat(
    objs=(dfs.set_index('ID') for dfs in info_dfs),
    axis=1,
    join='outer').reset_index().fillna(0)

But I get this error:

InvalidIndexError: Reindexing only valid with uniquely valued Index objects

I checked if individual dfs if they have duplicates but no one! Then I checked each df.index.is_unique and all come True I used ignore_index=True but the same error again! Not sure what is the problem here

Upvotes: 1

Views: 324

Answers (1)

Kanakorn Horsiritham
Kanakorn Horsiritham

Reputation: 98

IICU, df1 ... df6 share the ID column. And may be some customers don't have complete all information, no email for example.

Try pd.merge()

Like this,

import pandas as pd
info_dfs = (df1,df2,df3,df4,df5,df6)
final_df=None
for i in info_dfs:
   if final_df is None:
      final_df=i
   else:
      final_df=pd.merge(final_df,i,on='ID')
final_df

hope this help.

Upvotes: 1

Related Questions