CODEWITHSUNDEEP
pythonpandasdataframena
Anton Khodak
Anton Khodak

Reputation: 3696

Select rows from pandas data frame where specified columns are not all NaN

I have a Pandas DataFrame object data with columns 'a', 'b', 'c', ..., 'z'

I want to select all rows which satisfy the following condition: data in columns 'b', 'c', 'g' is not NaN simultaneously. I tried:

new_data = data[not all(np.isnan(value) for value in data[['b', 'c', 'g']])]

but it didn't work - throws an error:

Traceback (most recent call last):
  File "<input>", line 1, in <module>`
  File "<input>", line 1, in <genexpr>
 TypeError: Not implemented for this type

Upvotes: 2

Views: 1744

Answers (1)

eumiro
eumiro

Reputation: 212895

I want to select all rows, which qualify the following condition: data in columns 'b', 'c', 'g' is not NaN simultaneously.

Then you can use dropna:

new_data = data.dropna(how='all', subset=['b', 'c', 'g'])

using parameters:

how : {'any', 'all'}
    * any : if any NA values are present, drop that label
    * all : if all values are NA, drop that label
subset : array-like
    Labels along other axis to consider, e.g. if you are dropping rows
    these would be a list of columns to include

Upvotes: 1

Related Questions