CODEWITHSUNDEEP
python-3.xpandasdataframe
ah bon
ah bon

Reputation: 10061

Filter NaN rows for containing specific characters columns in Pandas

Given a small dataset as follow:

   *id *building  floor_number *floor_name
0    1         A           8.0          5F
1    2         A           4.0          4F
2    3         A           NaN          3F
3    4         A           NaN         NaN
4    5         A           NaN         NaN
5    6         B          14.0         17F
6    7         B          13.0         16F
7    8         B          20.0       world
8    9         B          13.0       hello
9   10         B          13.0         16F

I want to check if the containing * columns (selected_col as below), have NaN values, so I need to write a condition code for filtering NaN rows for selected_col.

selected_col = df.columns[df.columns.str.contains(pat = '^\*')]
...

How could I filter out rows as follows? Thanks.

   *id *building  floor_number *floor_name
3    4         A           NaN         NaN
4    5         A           NaN         NaN

EDIT: since I need to hightlight NaN cell for the columns with containing * (which means this column shouldn't have any NaNs), so I want to use np.where if possible.

np.where(conditions, None, 'contains NaNs for required columns')

Reference related: Highlight dataframe cells based on multiple conditions in Python

Upvotes: 1

Views: 214

Answers (2)

Sivaram Rasathurai
Sivaram Rasathurai

Reputation: 6383

A better way is BEEN_YO's answer Another way, you can do as follow

  1. Filter the columns using list compression that you need to check NaN values,
  2. Check with isna() method
cols = [col for col in df.columns if '*' in col]
out = df[df[cols].isna().all(1)]

Upvotes: 1

BENY
BENY

Reputation: 323386

Check with filter with any

out = df[df.filter(like='*').isna().any(1)]
Out[39]: 
   *id *building  floor_number *floor_name
3    4         A           NaN         NaN
4    5         A           NaN         NaN

Upvotes: 2

Related Questions