CODEWITHSUNDEEP
pythonnumpyscipysparse-matrix
Roland Deschain
Roland Deschain

Reputation: 2830

How to handle calculations on huge numpy array to avoid memory allocation error?

I need a negative identity matrix of size (62500 x 62500). declaring a normal identity matrix using numpy works like a charm:

eye = np.eye(62500, 62500)

However, doing something like this

negative_eye1 = np.negative(np.eye(62500, 62500))
# or
negative_eye2 = np.eye(62500, 62500) * -1

will result in this error

Unable to allocate array with shape (62500, 62500) and data type float64

The matrix is then used in a scipy.sparse.bmat() function, resulting in a csr-matrix, where memory won't be such an issue anymore.

How can I calculate this matrix?

Upvotes: 1

Views: 314

Answers (1)

kuzand
kuzand

Reputation: 9806

You can use scipy.sparse.eye (sparse matrix with ones on diagonal):

from scipy import sparse

negative_eye = -sparse.eye(62500, 62500)

Upvotes: 1

Related Questions