CODEWITHSUNDEEP
pythonpandasdataframeconcatenation
Vladimir Kolenov
Vladimir Kolenov

Reputation: 438

Pandas concat results are misaligned with NaNs

I had a dataframe for fitting ML algo with both categorial and numerical fatures. So I splitted this df and converted categorial features with one-hot encoding and now I have numpy matrix which I have to join back to df with numerical features so first row of numerical dataframe will get first matrix row as new features.

Given to documentation and results of googling 'join dataframes' command 

pd.concat([X_numerical, pd.DataFrame(numpy_matrix)], axis=1)

But I am getting this - like 'axis' param was ignored (I cleaned up all raw data from np.nan):

result example

What am I doing wrong? I expected to get resulting dataframe without these NaNs, with df rows concatenated one by one with matrix rows

Upvotes: 3

Views: 1576

Answers (1)

cs95
cs95

Reputation: 402553

I see a possible issue with index mis-alignment. The best thing to do is to reset the index:

df = pd.concat([
       X_numerical.reset_index(), 
       pd.DataFrame(numpy_matrix)
   ], axis=1)

Upvotes: 6

Related Questions