CODEWITHSUNDEEP
pythonnumpymemory-managementoptimizationscipy
Mayank Mandava
Mayank Mandava

Reputation: 61

Avoid creating new arrays as results for numpy/scipy operations?

For doing repeated operations in numpy/scipy, there's a lot of overhead because most operation return a new object.

For example

for i in range(100):
   x = A*x

I would like to avoid this by passing a reference to the operation, like you would in C

for i in range(100):
   np.dot(A,x,x_new) #x_new would now store the result of the multiplication
   x,x_new = x_new,x

Is there any way to do this? I would like this not for just mutiplication but all operations that return a matrix or a vector.

Upvotes: 4

Views: 1450

Answers (1)

EelkeSpaak
EelkeSpaak

Reputation: 2825

See Learning to avoid unnecessary array copies in IPython Books. From there, note e.g. these guidelines:

a *= b

will not produce a copy, whereas:

a = a * b

will produce a copy. Also, flatten() will copy, while ravel() only copies if necessary and returns a view otherwise (and thus should in general be preferred). reshape() also does not produce a copy, but returns a view.

Furthermore, as @hpaulj and @ali_m noted in their comments, many numpy functions support an out parameter, so have a look at the docs. From numpy.dot() docs:

out : ndarray, optional Output argument.

This must have the exact kind that would be returned if it was not used. In particular, it must have the right type, must be C-contiguous, and its dtype must be the dtype that would be returned for dot(a,b). This is a performance feature. Therefore, if these conditions are not met, an exception is raised, instead of attempting to be flexible.

Upvotes: 4

Related Questions