CODEWITHSUNDEEP
pythonnumpy
dagrun
dagrun

Reputation: 651

Get column number of elements that are greater than a threshold in 2D numpy array

I have a array like this and would like to get returned the column numbers for each row where the value is over the threshold of 0.6:

X = array([[ 0.16,  0.40,  0.61,  0.48,  0.20],
        [ 0.42,  0.79,  0.64,  0.54,  0.52],
        [ 0.64,  0.64,  0.24,  0.63,  0.43],
        [ 0.33,  0.54,  0.61,  0.43,  0.29],
        [ 0.25,  0.56,  0.42,  0.69,  0.62]])

Result would be: 

[[2],
[1, 2],
[0, 1, 3],
[2],
[3, 4]]

Is there a better way of doing this then by a double for-loop?

def get_column_over_threshold(data, threshold):
    column_numbers = [[] for x in xrange(0,len(data))]
    for sample in data:
        for i, value in enumerate(data):
            if value >= threshold:
                column_numbers[i].extend(i)
    return topic_predictions

Upvotes: 1

Views: 2510

Answers (2)

jmd_dk
jmd_dk

Reputation: 13090

For each row you can ask for the indices where the elements are greater than 0.6:

result = [where(row > 0.6) for row in X]

This performs the computation you want, but the format of result is somewhat inconvenient, since the result of where in this case is a tuple of size 1, containing a NumPy array with the indices. We can replace where with flatnonzero to get the array directly rather than the tuple. To obtain a list of lists, we explicitly cast this array to a list:

result = [list(flatnonzero(row > 0.6)) for row in X]

(In the code above I assume you have used from numpy import *)

Upvotes: 2

Divakar
Divakar

Reputation: 221504

Use np.where to get row, col indices and then use those with np.split to get list of column indices as arrays output -

In [18]: r,c = np.where(X>0.6)

In [19]: np.split(c,np.flatnonzero(r[:-1] != r[1:])+1)
Out[19]: [array([2]), array([1, 2]), array([0, 1, 3]), array([2]), array([3, 4])]

To make it more generic which would handle rows without any match, we could loop through the column indices obtained from np.where and assign into an initialized array, like so -

def col_indices_per_row(X, thresh):
    mask = X>thresh
    r,c = np.where(mask)
    out = np.empty(len(X), dtype=object)
    grp_idx = np.r_[0,np.flatnonzero(r[:-1] != r[1:])+1,len(r)]
    valid_rows = r[np.r_[True,r[:-1] != r[1:]]]
    for (row,i,j) in zip(valid_rows,grp_idx[:-1],grp_idx[1:]):
        out[row] = c[i:j]     
    return out

Sample run -

In [92]: X
Out[92]: 
array([[0.16, 0.4 , 0.61, 0.48, 0.2 ],
       [0.42, 0.79, 0.64, 0.54, 0.52],
       [0.1 , 0.1 , 0.1 , 0.1 , 0.1 ],
       [0.33, 0.54, 0.61, 0.43, 0.29],
       [0.25, 0.56, 0.42, 0.69, 0.62]])

In [93]: col_indices_per_row(X, thresh=0.6)
Out[93]: 
array([array([2]), array([1, 2]), None, array([2]), array([3, 4])],
      dtype=object)

Upvotes: 1

Related Questions