CODEWITHSUNDEEP
pythonpandas
user642897
user642897

Reputation: 9631

difference between "!=" and "not_equal" in pandas

When selecting from a DataFrame:

Using not_equal gives the correct result:

d[not_equal(d.something,None)]

However using != does not:

d[d.something != None]

Why is this?

update, d.something dtype('float64'), the same is also the case if I try to select NaN values d[d.something != nan]

Upvotes: 2

Views: 4080

Answers (1)

D. A.
D. A.

Reputation: 3509

I'm going to answer a bit about "the same is also the case if I try to select NaN values d[d.something != nan]"

You need to be aware that NaN doesn't compare as equal to another NaN:

In [40]: numpy.NaN == numpy.NaN
Out[40]: False

In [41]: numpy.NaN != numpy.NaN
Out[41]: True

This may seem backwards. However, when you think of the first along the lines of "if it isn't a number, it can't be equal to anything", it gets more clear. == will always return False with NaN as either side. The, if you interpret a != b as not (a == b), the second makes sense too. That could explain part of the problem. Your d[d.something != NaN] will always return d.

I will look more into the other issue by digging into the code.

Upvotes: 1

Related Questions