CODEWITHSUNDEEP
pythonnumpynan
usethedeathstar
usethedeathstar

Reputation: 2309

making numpy.nanargmin return nan if column is all nan

Is it possible to use numpy.nanargmin, so that it returns numpy.nan, on columns where there are only nans in them. Right now, it raises a ValueError, when that happens. And i cant use numpy.argmin, since that will fail when there are only a few nans in the column.

http://docs.scipy.org/doc/numpy/reference/generated/numpy.nanargmin.html says that the ValueError is raised for all-nan slices. In that case, i want it to return numpy.nan (just to further mask the "non-data" with nans)

this next bit does this, but is super-slow and not really pythonic:

for i in range(R.shape[0]):
    bestindex = numpy.nanargmin(R[i,:])
    if(numpy.isnan(bestindex)):
        bestepsilons[i]=numpy.nan
    else:
        bestepsilons[i]=epsilon[bestindex]

This next bit works too, but only if no all-nan columns are involved:

ar = numpy.nanargmin(R, axis=1)
bestepsilons = epsilon[ar]

So ideally i would want this last bit to work with all-nan columns as well

Upvotes: 6

Views: 4499

Answers (3)

Ivano Barletta
Ivano Barletta

Reputation: 107

I had a similar problem with an array of shape (nz,ny,nx) with some slices [:,j,i] totally filled with NaNs. In my case I needed argmax index along axis=0 and if I do

np.nanargmax(array,axis=0)

I get "ValueError: All-NaN slice encountered"

Considering that I'm not interested into argmax for slices made all of NaNs, as a workaround I filled the NaNs with zeros

mask = np.isnan(array)
array[mask] = 0
idx2d = np.argmax(array,axis=0)

This gives the indexes of max of array along axis=0. The idx2d array can be re-masked again

idx2d = np.ma.masked_where(mask[0],idx2d)

If you seek for np.argmin you can do the same steps but setting your array to a Huge number instead of 0.

Upvotes: 0

usethedeathstar
usethedeathstar

Reputation: 2309

Found a solution:

# makes everything nan to start with
bestepsilons1 = numpy.zeros(R.shape[0])+numpy.nan 
# finds the indices where the entire column would be nan, so the nanargmin would raise an error
d0 = numpy.nanmin(R, axis=1) 
# on the indices where we do not have a nan-column, get the right index with nanargmin, and than put the right value in those points
bestepsilons1[~numpy.isnan(d0)] = epsilon[numpy.nanargmin(R[~numpy.isnan(d0),:], axis=1)]

This basically is a workaround, by only taking the nanargmin on the places where it will not give an error, since at those places we want the resulting index to be a nan anyways

Upvotes: 3

ev-br
ev-br

Reputation: 26040

>>> def _nanargmin(arr, axis):
...    try:
...       return np.nanargmin(arr, axis)
...    except ValueError:
...       return np.nan

Demo:

>>> a = np.array([[np.nan]*10, np.ones(10)])
>>> _nanargmin(a, axis=1)
nan
>>> _nanargmin(a, axis=0)
array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1])

Anyway, it's unlikely to be what you want. Not sure what exactly you are after. If all you want is to filter away the nans, then use boolean indexing:

>>> a[~np.isnan(a)]
array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.])
>>> np.argmin(_)
0

EDIT2: Looks like you're after the masked arrays:

>>> a = np.vstack(([np.nan]*10, np.arange(10), np.arange(11, 1, -1)))
>>> a[2, 4] = np.nan
>>> m = np.ma.masked_array(a, np.isnan(a))
>>> np.argmin(m, axis=0)
array([1, 1, 1, 1, 1, 1, 2, 2, 2, 2])
>>> np.argmin(m, axis=1)
array([0, 0, 9])

Upvotes: 5

Related Questions