CODEWITHSUNDEEP
pythonpandasdataframeconcatenation
Starbucks
Starbucks

Reputation: 1568

Concatenate two dataframes without loosing columns

I've tried concatenating two dataframes using pd.concat([df1, df2], axis = 1), however this does not keep all of the column names in df1. 

import pandas as pd
from datetime import timedelta
df = pd.DataFrame({
        'date': ['2001-02-01','2001-02-02','2001-02-03', '2001-02-04'],
        'future_date': ['2001-02-01','2001-02-02','2001-02-03', '2001-02-04'],
        'x Value': [100, 200, 300, 400],
'x1 Value': [100, 200, 400, 400],
'x2 Value': [100, 300, 300, 400],
'xN Value': [200, 200, 300, 400]})

df2 = (df1['future_date'] + pd.Timedelta(5, unit='days')).to_frame()
pd.concat([df1, df2], axis = 1).tail(10)

When I return pd.concat([df1, df2], axis = 1).tail(10) I only get the future_date column values.

Desired dataframe:

    future_date date       x Value  x1 Value    x2 Value    xN Value
0   2001-02-01  2001-02-01 100  100 100 200
1   2001-02-02  2001-02-02 200  200 300 200
2   2001-02-03  2001-02-03 300  400 300 300
3   2001-02-04  2001-02-04 400  400 400 400
4   2001-02-05  
5   2001-02-06
6   2001-02-07
7   2001-02-08

I've also tried

new_df = pd.concat([df2, df1(columns=df1.columns.tolist())], ignore_index=True)

TypeError: 'DataFrame' object is not callable

Upvotes: 0

Views: 57

Answers (1)

YOLO
YOLO

Reputation: 21719

You need to create the missing columns first.

for col in df.columns:
    if col not in df2.columns:
        df2[col] = None

df2 = pd.concat([df, df2], ignore_index=True)

         date future_date x Value x1 Value x2 Value xN Value
0  2001-02-01  2001-02-01     100      100      100      200
1  2001-02-02  2001-02-02     200      200      300      200
2  2001-02-03  2001-02-03     300      400      300      300
3  2001-02-04  2001-02-04     400      400      400      400
4        None  2001-02-06    None     None     None     None
5        None  2001-02-07    None     None     None     None
6        None  2001-02-08    None     None     None     None
7        None  2001-02-09    None     None     None     None

Upvotes: 2

Related Questions