CODEWITHSUNDEEP
pythonarrayspython-3.xnumpy
Py-ser
Py-ser

Reputation: 2098

Delete nan AND corresponding elements in two same-length array

I have two lists of the same length that I can convert into array to play with the numpy.stats.pearsonr method. Now, some of the elements of these lists are nan, and can thus not be used for that method. The best thing to do in my case is to remove those elements, and the corresponding element in the other list. Is there a practical and pythonic way to do it? Example: I have

[1 2 nan 4 5 6 ] and [1 nan 3 nan 5 6]

and in the end I need

[1 5 6 ]

[1 5 6 ]

(here the number are representative of the position/indices, not of the actual numbers I am dealing with). EDIT: The tricky part here is to have both lists/arrays without nans in one array AND elements corresponding to nans in the other, and vice versa. Although it can certainly be done by manipulating the arrays, I am sure there is a clear and not overcomplicated way to do it in a pythonic way.

Upvotes: 4

Views: 1229

Answers (1)

Antti Haapala -- Слава Україні
Antti Haapala -- Слава Україні

Reputation: 133929

The accepted answer to proposed duplicate gets you half-way there. Since you're using Numpy already you should make these into numpy arrays. Then you should generate an indexing expression, and then use it to index these 2 arrays. Here indices will be a new array of bool of same shape where each element is True iff not (respective element in x is nan or respective element in y is nan):

>>> x
array([  1.,   2.,  nan,   4.,   5.,   6.])
>>> y
array([  1.,  nan,   3.,  nan,   5.,   6.])
>>> indices = np.logical_not(np.logical_or(np.isnan(x), np.isnan(y)))
>>> x = x[indices]
>>> y = y[indices]
>>> x
array([ 1.,  5.,  6.])
>>> y
array([ 1.,  5.,  6.])

Notably, this works for any 2 arrays of same shape.

P.S., if you know that the element type in the operand arrays is boolean, as is the case for arrays returned from isnan here, you can use ~ instead of logical_not and | instead of logical_or: indices = ~(np.isnan(x) | np.isnan(y))

Upvotes: 7

Related Questions