CODEWITHSUNDEEP
pythonnumpyvectorization
user1581390
user1581390

Reputation: 1998

How to Vectorize This Matrix Operation?

(I asked a similar question before but this is a different operation.)

I have 2 arrays of boolean masks and I am looking to calculate an operation on every combination of two masks.

The slow version

N = 10000
M = 580
masksA = np.array(np.random.randint(0,2, size=(N,M)), dtype=np.bool)
masksB = np.array(np.random.randint(0,2, size=(N,M)), dtype=np.bool)

result = np.zeros(shape=(N,N), dtype=np.float)
for i in range(N):
    for j in range(N):
        result[i,j] = np.float64(np.count_nonzero(np.logical_and(masksA[i,:],masksB[j,:]))) / M

Upvotes: 2

Views: 72

Answers (1)

Divakar
Divakar

Reputation: 221564

It seems the first input would be masksA as the question text reads - "operation on every combination of two masks".

We can use matrix-multiplication to solve it, like so -

result = masksA.astype(np.float).dot(masksB.T)/M

Alternatively, use lower precision np.float32 for dtype conversion for faster computations. Since, we are counting, it should be fine with lower precision.

Timings -

In [5]: N = 10000
   ...: M = 580
   ...: 
   ...: np.random.seed(0)
   ...: masksA = np.array(np.random.randint(0,2, size=(N,M)), dtype=np.bool)
   ...: masksB = np.array(np.random.randint(0,2, size=(N,M)), dtype=np.bool)

In [6]: %timeit masksA.astype(np.float).dot(masksB.T)
1.87 s ± 50.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [7]: %timeit masksA.astype(np.float32).dot(masksB.T)
1 s ± 7.93 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Upvotes: 3

Related Questions