CODEWITHSUNDEEP
pythonarrayspandasdataframenan
Victor Murcia
Victor Murcia

Reputation: 25

Effiiciently remove row from pandas dataframe with NaN in array

I have a pandas dataframe where columns labeled e, f, and g contain numpy arrays corresponding to RGB values.

a b c d e f g
0 0 62 0 [89, 86, 78] [178, 171, 163] [63, 58, 50]
1 0 68 0 [99, 94, 74] [121, 113, 88] [113, 108, 79]
2 0 68 1 [63, 7, 7] [14, 9, 7] [56, 42, 26]

Some of the arrays however contain NaN values as part of their composition and I would like to remove the rows that contain them. I have a routine that does exactly this but I was wondering if there was a more efficient way to accomplish it.

The code below does what I want but I was wondering if there is a faster/better/cleaner solution.

row,col = df.shape
for r in range(row):
   rv = df['e'][r][0]
   if type(rv) != np.uint32:
       df = df.drop(r)

I tried implementing variations of the isnull and isna methods but haven't been successful yet. I think that the issue may be due to the fact that I have arrays as the data elements.

Any suggestions would be greatly appreciated!

Thanks!

Upvotes: 1

Views: 62

Answers (1)

BENY
BENY

Reputation: 323226

You can try with applymap

df = df[~df.applymap(lambda x : np.isnan(x).any()).any(1)]

Upvotes: 2

Related Questions