CODEWITHSUNDEEP
pythonpandasnumpy
pistacchio
pistacchio

Reputation: 58863

Pandas: drop rows with nan-valued index

Given the following pandas DataFrame where some indices are NaN, how to drop the third and eight row since their index is NaN? Thanks

import pandas as pd
import numpy  as np

data    = list('abcdefghil')
indices = [0, 1, np.nan, 3, 4, 5, 6, np.nan, 8, 9]

df = pd.DataFrame(data, index=indices, columns=['data'])

Upvotes: 5

Views: 4549

Answers (3)

EdChum
EdChum

Reputation: 393863

You can call dropna on the index:

In[68]:
df.loc[df.index.dropna()]

Out[68]: 
    data
0.0    a
1.0    b
3.0    d
4.0    e
5.0    f
6.0    g
8.0    i
9.0    l

Note that the presence of NaN makes the index dtype float, to change it to int cast the type:

In[70]:
df = df.loc[df.index.dropna()]
df.index = df.index.astype(int)
df

Out[70]: 
  data
0    a
1    b
3    d
4    e
5    f
6    g
8    i
9    l

You can also call notnull on the index would also work (somehow undocumented)

In[71]:
df = df.loc[df.index.notnull()]
df.index = df.index.astype(int)
df

Out[71]: 
  data
0    a
1    b
3    d
4    e
5    f
6    g
8    i
9    l

there is also isna:

In[78]:
df.loc[~df.index.isna()]

Out[78]: 
    data
0.0    a
1.0    b
3.0    d
4.0    e
5.0    f
6.0    g
8.0    i
9.0    l

and the more readable inverse notna:

In[79]:
df.loc[df.index.notna()]

Out[79]: 
    data
0.0    a
1.0    b
3.0    d
4.0    e
5.0    f
6.0    g
8.0    i
9.0    l

As commented by @jpp you can use the top-level notnull also:

In[80]:
df.loc[pd.notnull(df.index)]

Out[80]: 
    data
0.0    a
1.0    b
3.0    d
4.0    e
5.0    f
6.0    g
8.0    i
9.0    l

There is also top-level isna, notna, and isnull but I'm not going to display those, you can check the docs

Upvotes: 9

Benjamin Breton
Benjamin Breton

Reputation: 1547

You can use the following: 

 df = df[df.index.isnull() == False]

You might want to reset the index after

Upvotes: 2

jpp
jpp

Reputation: 164613

Using np.isnan and taking the negative:

res = df[~np.isnan(df.index)]

print(res)

    data
0.0    a
1.0    b
3.0    d
4.0    e
5.0    f
6.0    g
8.0    i
9.0    l

Upvotes: 1

Related Questions