CODEWITHSUNDEEP
pythonnumpyabstract-data-typecomplex-data-types
psitae
psitae

Reputation: 103

Change the data type of one element in a matrix

I'm looking to implement a hardware-efficient multiplication of a list of large matrices (on the order of 200,000 x 200,000). The matrices are very nearly the identity matrix, but with some elements changed to irrational numbers.

In an effort to reduce the memory footprint and make the computation go faster, I want to store the 0s and 1s of the identity as single bytes like so.

import numpy as np
size = 200000
large_matrix = np.identity(size, dtype=uint8)

and just change a few elements to a different data type.

import sympy as sp

# sympy object
irr1 = sp.sqrt(2)    

# float
irr2 = e               

large_matrix[123456, 100456] = irr1
large_matirx[100456, 123456] = irr2

Is is possible to hold only these elements of the matrix with a different data type, while all the other elements are still bytes? I don't want to have to change everything to a float just because I need one element to be a float.

-----Edit-----

If it's not possible in numpy, then how can I find a solution without numpy?

Upvotes: 0

Views: 385

Answers (2)

Mario García
Mario García

Reputation: 182

Maybe you can have a look at the SciPy's Coordinate-based sparse matrix. In that case SciPy creates a sparse matrix (optimized for such large empty matrices) and with its coordinate format you can access and modify the data as you intend.

From its documentation:

>>> from scipy.sparse import coo_matrix
>>> # Constructing a matrix using ijv format
>>> row  = np.array([0, 3, 1, 0])
>>> col  = np.array([0, 3, 1, 2])
>>> data = np.array([4, 5, 7, 9])
>>> m = coo_matrix((data, (row, col)), shape=(4, 4))
>>> m.toarray()
array([[4, 0, 9, 0],
       [0, 7, 0, 0],
       [0, 0, 0, 0],
       [0, 0, 0, 5]])

It does not create a matrix but a set of coordinates with values, which takes much less space than just filling a matrix with zeros.

>>> from sys import getsizeof
>>> getsizeof(m)
56
>>> getsizeof(m.toarray())
176

Upvotes: 2

Bruno Mello
Bruno Mello

Reputation: 4618

By definition, NumPy arrays only have one dtype. You can see in the NumPy documentation:

A numpy array is homogeneous, and contains elements described by a dtype object. A dtype object can be constructed from different combinations of fundamental numeric types.

Further reading: https://docs.scipy.org/doc/numpy/reference/generated/numpy.dtype.html

Upvotes: 0

Related Questions