CODEWITHSUNDEEP
python-3.xnumpy
Sushmita
Sushmita

Reputation: 85

Increase speed of numpy nested for loop for huge matrix

I have a python code, which has nested for loop, for a huge matrix. Here rho0, alp0, bet0 are constant. and E is a symmetric matrix. How can i increase is the speed of for loop?

N = 20000
Q = np.zeros(shape=(N,N))               
for i in tqdm(range(0,N)):
    for j in range(0,N):
        nom    = rho0*alp0**E[i,j]*(1-alp0)**(M-E[i,j]);
        dnom   = nom + ( (1-rho0)*bet0**E[i,j]*(1-bet0)**(M-E[i,j]));
        Q[i,j] = nom/dnom;

Upvotes: 0

Views: 125

Answers (2)

hpaulj
hpaulj

Reputation: 231355

If everything is scalars constants except E, then this should work

    nom    = rho0*alp0**E*(1-alp0)**(M-E)
    dnom   = nom + ( (1-rho0)*bet0**E*(1-bet0)**(M-E))
    Q = nom/dnom

A numpy array E can be multiplied, subtracted etc as a whole. The key to speed in numpy is to work with the compiled whole-array methods (and operators) where possible. Those methods do iterate, but they do so at compiled speeds, rather than the (much) slower Python speed.

There are tools for compiling larger expressions, like numba, but work with the core numpy for a start. Do take some time read numpy basics; it will save you time later.

Upvotes: 0

Arya McCarthy
Arya McCarthy

Reputation: 8829

You can tremendously speed your code up by replacing Python looping with hardware-accelerated vector operations and tight C or Fortran loops provided by numpy.

(You didn't define several variables, so I took the liberty of defining them.)

import numpy as np
from tqdm import tqdm

# Set these to the correct values.
rho = 0.6
alp0 = 1.4
bet0 = 2.5
M = 3.6
_E = np.random.rand(N, N)
E = _E + _E.T  # symmetric matrix.

# Look, all 20000 * 20000 values computed in one move!
nom = rho0 * alp0 ** E * (1 - alp0) ** (M - E)  # Shape is 20000x20000
dnim = nom + ((1 - rho0) * bet0 ** E * (1 - bet0) ** (M - E))  # Shape is 20000x20000
Q = nom / dnom

Upvotes: 1

Related Questions