CODEWITHSUNDEEP
pythonpandasdataframerowsnan
Vityata
Vityata

Reputation: 43585

Count number of rows with NaN in a pandas DataFrame?

Having the following running code:

import datetime as dt
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

my_funds = [1, 2, 5, 7, 9, 11]
my_time = ['2020-01', '2019-12', '2019-11', '2019-10', '2019-09', '2019-08']
df = pd.DataFrame({'TIME': my_time, 'FUNDS':my_funds})

for x in range(2,3):
    df.insert(len(df.columns), f'x**{x}', df["FUNDS"]**x)

df = df.replace([1, 7, 9, 25],float('nan'))

print(df.isnull().values.ravel().sum())   #5 (obviously counting NaNs in total)
print(sum(map(any, df.isnull())))         #3 (I guess counting the NaNs in the left column)

I am getting the dataframe below. I want to get the count of the total rows, with 1 or more NaN, which in my case is 4, on rows - [0, 2, 3, 4].

enter image description here

Upvotes: 1

Views: 3588

Answers (3)

ansev
ansev

Reputation: 30920

New option Series.clip

to take one when there is more than one NaN per row

df.isna().sum(axis=1).clip(upper=1).sum()
#4

Upvotes: 1

Golden
Golden

Reputation: 417

Another option:

nan_rows = len(df[df["FUNDS"].isna() | df["x**2"].isna()])

Upvotes: 3

jezrael
jezrael

Reputation: 862741

Use:

print (df.isna().any(axis=1).sum())
4

Explanation: First compare missing values by DataFrame.isna:

print (df.isna())
    TIME  FUNDS   x**2
0  False   True   True
1  False  False  False
2  False  False   True
3  False   True  False
4  False   True  False
5  False  False  False

And test if at least per rows is True by DataFrame.any:

print (df.isna().any(axis=1))
0     True
1    False
2     True
3     True
4     True
5    False
dtype: bool

And last count Trues by sum.

Upvotes: 7

Related Questions