CODEWITHSUNDEEP
pythonpython-3.xnumpyvectormatrix-multiplication
user2863620
user2863620

Reputation: 643

Very large matrix vector product

I have a large symmetric matrix A of dimensions (N, N) (N is about twenty million), and for sure I cannot store this matrix (50% components of A are zeros).

But every component A[i, j] is explicitly known: A[i, j] = f(i, j). For example A[i, j] = cos(i)*cos(j).

I need to multiply this matrix with a vector of length N. What is "doable" way to do that on a machine of 64 cores, 128GB of RAM?

Upvotes: 2

Views: 498

Answers (1)

pss
pss

Reputation: 744

If you have a way to compute elements of matrix on the fly there is no need to store whole matrix in memory. Also each element of result vector in independent of each other so you can run as many parallel workers as you want.

The only optimization of algorithm I can think of is take into consideration that f(i, j) = cos(i)*cos(j) is symmetric function (f(i, j) = f(j, i)). But that's if this is your real function.

Also check numpy and Cython for much faster computations in Python as pure Python can be a little slow for this kind of job.

Upvotes: 1

Related Questions