CODEWITHSUNDEEP
pythonpython-2.7pandasnan
sds
sds

Reputation: 60014

Pandas: nan->None

pandas.DataFrame.to_dict converts nan to nan and null to None. As explained in Python comparison ignoring nan this is sometimes suboptimal.

Is there a way to convert all nans to None? (either in pandas or later on in Python)

E.g., 

>>> df = pd.DataFrame({"a":[1,None],"b":[None,"foo"]})
>>> df
     a     b
0  1.0  None
1  NaN   foo
>>> df.to_dict()
{'a': {0: 1.0, 1: nan}, 'b': {0: None, 1: 'foo'}}

I want

{'a': {0: 1.0, 1: None}, 'b': {0: None, 1: 'foo'}}

instead.

Upvotes: 8

Views: 5487

Answers (3)

markling
markling

Reputation: 1384

I found that the accepted answer did not work, but this did:

df.replace([np.nan], [None]).to_dict('records')

I don't know why. I can say at least that all fields of the df that appeared to have na values in them did verify as such by checking them with df.isna().

I got the solution from here.

Upvotes: 0

BENY
BENY

Reputation: 323306

import pandas as pd

df = pd.DataFrame({"a":[1,None],"b":[None,"foo"]})
df.where((pd.notnull(df)), None)
Out[850]: 
      a     b
0     1  None
1  None   foo
df.where((pd.notnull(df)), None).to_dict()
Out[851]: {'a': {0: 1.0, 1: None}, 'b': {0: None, 1: 'foo'}}

Upvotes: 10

cs95
cs95

Reputation: 402673

Initialise as an object DataFrame (at your peril...):

df = pd.DataFrame({"a":[1,None],"b":[None,"foo"]}, dtype=object)    
df

      a     b
0     1  None
1  None   foo

In the first column, pandas attempts to infer the dtype, and guesses float. You can prevent that by forcing it to remain object thereby suppressing any type of conversion at all.

Upvotes: 3

Related Questions