CODEWITHSUNDEEP
pythonnumpypandas
Luis Miguel
Luis Miguel

Reputation: 5127

strange behavior: apparently a numpy nan in a pandas dataframe showing up as dtype('float64')

I am working on a dataframe that has many custom functions, libraries, and calculations. After doing some critical calcs, I noticed some errors in calculations that should have returned a float:

enter image description here

To inspect one of the calculations, I do the following.

dFA.loc['20101120']['variable x']

which returns (in small caps)

nan

Then, to confirm that this thing is what looks like a weird (small caps) numpy.nan (True or False) I do:

dFA.loc['20101120']['variable x'] == np.nan

Which returns: 

False

Then I do:

dFA.loc['20101120']['variable x'].dtype

Which returns:

dtype('float64')

Also:

dFA.loc['20101120']['variable x'] > 1000
False

Also:

dFA.loc['20101120']['variable x'] < 1000
False

Upvotes: 2

Views: 358

Answers (2)

timgeb
timgeb

Reputation: 78650

All comparisons with np.nan evaluate to False by definition.

>>> np.nan == np.nan
False
>>> np.nan <= 1
False
>>> np.nan > 1
False

np.nan is a float: 

>>> np.nan.__class__
<type 'float'>

... just a very special one.

Upvotes: 2

Ignacio Vazquez-Abrams
Ignacio Vazquez-Abrams

Reputation: 798526

dFA.loc['20101120']['variable x'] == np.nan

Oops. NaN is never equal to NaN.

np.isnan(dFA.loc['20101120']['variable x'])

Upvotes: 3

Related Questions