CODEWITHSUNDEEP
pandasdataframereplacenannonetype
AdamNYC
AdamNYC

Reputation: 20415

Replace None with NaN in pandas dataframe

I have table x:

        website
0   http://www.google.com/
1   http://www.yahoo.com
2   None

I want to replace python None with pandas NaN. I tried:

x.replace(to_replace=None, value=np.nan)

But I got:

TypeError: 'regex' must be a string or a compiled regular expression or a list or dict of strings or regular expressions, you passed a 'bool'

How should I go about it?

Upvotes: 194

Views: 344150

Answers (7)

igorkf
igorkf

Reputation: 3565

This solution is straightforward because can replace the value in all the columns easily.
You can use a dict:

import pandas as pd
import numpy as np

df = pd.DataFrame([[None, None], [None, None]])
print(df)
      0     1
0  None  None
1  None  None

# replacing
df = df.replace({None: np.nan})
print(df)
    0   1
0 NaN NaN
1 NaN NaN

Upvotes: 8

Mitzi
Mitzi

Reputation: 2811

Its an old question but here is a solution for multiple columns:

values = {'col_A': 0, 'col_B': 0, 'col_C': 0, 'col_D': 0}
df.fillna(value=values, inplace=True)

For more options, check the docs:

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.fillna.html

Upvotes: 1

Guillaume Jacquenot
Guillaume Jacquenot

Reputation: 11707

You can use DataFrame.fillna or Series.fillna which will replace the Python object None, not the string 'None'.

import pandas as pd
import numpy as np

For dataframe:

df = df.fillna(value=np.nan)

For column or series:

df.mycol.fillna(value=np.nan, inplace=True)

Upvotes: 276

Ashish Sharma
Ashish Sharma

Reputation: 9

DataFrame['Col_name'].replace("None", np.nan, inplace=True)

Upvotes: 0

Doubledown
Doubledown

Reputation: 468

If you use df.replace([None], np.nan, inplace=True), this changed all datetime objects with missing data to object dtypes. So now you may have broken queries unless you change them back to datetime which can be taxing depending on the size of your data.

If you want to use this method, you can first identify the object dtype fields in your df and then replace the None:

obj_columns = list(df.select_dtypes(include=['object']).columns.values)
df[obj_columns] = df[obj_columns].replace([None], np.nan)

Upvotes: 8

Nickolai
Nickolai

Reputation: 1758

Here's another option:

df.replace(to_replace=[None], value=np.nan, inplace=True)

Upvotes: 36

Maz Izadi
Maz Izadi

Reputation: 337

The following line replaces None with NaN:

df['column'].replace('None', np.nan, inplace=True)

Upvotes: 28

Related Questions