CODEWITHSUNDEEP
pythonarraysperformancenumpy
pd shah
pd shah

Reputation: 1406

fastest way to search in 3D array in numpy

there is an array like:

array([
   [[X1, Y1, 80, 13, 14],
    [17, 54, 47, 67, 45],
    [11, 21, 67, 45, 11],
    [15, 75, 17, 69, 12]],

   [[X2, Y2, 94, 51, 10],
    [64, 81, 90, 80, 67],
    [78, 73, 19,  6, 57],
    [46, 18, 83, 26, 56]],

    ...

   [[Xn, Yn, 32,  3, 82],
    [47, 69, 63, 20, 10],
    [16, 48,  7, 96, 11],
    [95, 30, 45, 65, 42]] ])

the goal is to group by every X's and Y's then check a condition in every group. to do this I write code:

x=npy.random.randint(100,size=(5,4,5))
array([[[36, 47, 80, 13, 14],
        [17, 54, 47, 67, 45],
        [11, 21, 67, 45, 11],
        [15, 75, 17, 69, 12]],

       [[50, 17, 94, 51, 10],
        [64, 81, 90, 80, 67],
        [78, 73, 19,  6, 57],
        [46, 18, 83, 26, 56]],

       [[37, 98, 32,  3, 82],
        [47, 69, 63, 20, 10],
        [16, 48,  7, 96, 11],
        [95, 30, 45, 65, 42]],

       [[59, 36, 38, 95,  3],
        [67, 62, 61, 64, 15],
        [49, 45, 24, 85, 24],
        [23, 89, 23, 63, 14]],

       [[49, 26, 70, 83, 99],
        [16, 94, 78, 43, 33],
        [95,  4, 39, 91, 56],
        [94, 26, 71, 32, 55]]])

for i in range(x.shape[1]):
  for j in range(x.shape[2]):
    npy.any(x[0:3,i,j]>80)

False
True
True
False
True
False
True
True
False
False
False
False
False
True
False
True
False
True
False
False

but because the array that I work with, is very big this way is not efficient to me. is there any better way to write? is it possible to remove for loops? what is the fastest way to this ?

Upvotes: 0

Views: 1277

Answers (2)

Daniel F
Daniel F

Reputation: 14399

If you're going to be doing a lot of groups (i.e. n is large compared to the size of the groups) you can even use logical_or.reduceat to do it all at once:

def groupSearch(x, groupSize, c = lambda y: y > 80, axis = 0):
    slices =  np.arange(0 , x.shape[axis] , groupSize)
    return np.logical_or.reduceat(c(x), slices, axis)

Upvotes: 0

Divakar
Divakar

Reputation: 221514

IIUC you could simply slice the first three elements along first axis, compare against the threshold and then look for ANY match along the first axis -

(x[0:3] > 80).any(axis=0)

Upvotes: 4

Related Questions