CODEWITHSUNDEEP
pythonpandas
V.Nouri
V.Nouri

Reputation: 277

Why pd.concat() of two dataframe leads to FutureWarning: Behavior when concatenating bool-dtype?

List item

I want to concatenate two dataframe with pd.concat() as below:

if (not df_1.empty) | (not df_2.empty):
    new_df= pd.concat([df_1, df_2])

It returns the following warning:

FutureWarning: Behavior when concatenating bool-dtype and numeric-dtype arrays is deprecated; in a future version these will cast to object dtype (instead of coercing bools to numeric values). To retain the old behavior, explicitly cast bool-dtype arrays to numeric dtype.

I have read this question as well, but here I have a DataFrame which contains str and different types of numbers (int, float). What should I do in this case?

This is the sample data of each dataframe: df_1:

dateTime entryRate stop limit amount stdLotds currencyName Buy
3/11/2022 11:24 1.31006 0 0 5000 0.05 GBPUSD True
3/11/2022 11:24 1.31007 0 0 1000 0.01 GBPUSD False
3/11/2022 11:11 1.79134 0 1.78448 2000 0.02 GBPAUD True

df_2:

dateTime entryRate stop limit amount stdLotds currencyName Buy
3/14/2022 10:24 1.31012 0 0 5000 0.05 GBPUSD False
3/11/2022 12:25 1.31017 0 0 3000 0.09 EURUSD False
3/14/2022 10:00 1.79114 0 1.78448 2000 0.03 AUDCAD True

Upvotes: 9

Views: 10195

Answers (4)

Georgie
Georgie

Reputation: 118

More dynamically:

ListMyDfs = ['df_1', 'df_2', 'df_3']
 
df = pd.DataFrame()
 
for MyDf in ListMyDfs:
    if globals()[MyDf].shape[0] > 0:
        df = pd.concat([df, globals()[MyDf]], axis=0)

Upvotes: 0

Junior2691
Junior2691

Reputation: 1

Yes, it does happen, despite null values not being present in the original data and even after applying fillna(0) operation. I think using Warnings Module might help. You may try the following:

warnings.simplefilter('ignore', 'DeprecationWarning')```

Upvotes: 0

Eelco van Vliet
Eelco van Vliet

Reputation: 1238

It should be

if (not df_1.empty) & (not df_2.empty):
    new_df= pd.concat([df_1, df_2])

i.e. and (&) in stead of or (|)

Upvotes: 3

Emil Jansson
Emil Jansson

Reputation: 159

I got this warning in an unexpected location when concatenating a list of dataframes that all contained the same columns with consistent datatypes across frames. The reason the warning appeared was that some frames had 0 rows and therefore were not assigned datatypes in the same way as the other frames. I stopped receiving the error after filtering out empty dataframes before concatenating.

Upvotes: 5

Related Questions