CODEWITHSUNDEEP
pythonpython-2.7pandas
user2242044
user2242044

Reputation: 9213

df.loc filtering doesn't work with None values

Why does this filtering not work when the filter is Project ID == None? I also noticed is None rather than == None returns KeyError: False 

import pandas as pd
df = pd.DataFrame(data = [['Project1', 'CT', 800], [None, 3, 1000], ['Project3', 'CA', 20]], columns=['Project ID', 'State', 'Cost'])

print df.loc[df['Project ID'] == 'Project1'].values
print df.loc[df['Project ID'] == None].values

output:

[['Project1' 'CT' 800L]]
[]

Upvotes: 7

Views: 4979

Answers (2)

EdChum
EdChum

Reputation: 394101

You have to use isnull for this:

In [3]:

df[df['Project ID'].isnull()]
Out[3]:
  Project ID State  Cost
1       None     3  1000

Or use apply:

In [5]:

df.loc[df['Project ID'].apply(lambda x: x is None)]
Out[5]:
  Project ID State  Cost
1       None     3  1000

Upvotes: 6

DeepSpace
DeepSpace

Reputation: 81614

Just to elaborate, it doesn't work because pandas use np.nan, and:

print np.nan == np.nan   # False
print np.nan == None     # False
print np.isnan(np.nan)   # True

Upvotes: 1

Related Questions