CODEWITHSUNDEEP
pythonnumpy
theta
theta

Reputation: 25601

Memory error while slicing an array

I have object d connected to h5 dataset:

>>> data = d[:, :, 0].astype(np.float32)
>>> data.shape
(17201, 10801)
>>> data[data==-32768] = data[data>0].min()
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
MemoryError

Can I do some other slicing trick to avoid this error?

Upvotes: 3

Views: 720

Answers (1)

theta
theta

Reputation: 25601

OK, I'm writing answer myself, as there is acceptable solution, gained after @mgilson questioned data type.

If data allows, memory error can be avoided by using simpler data type while operating on array. Considering initial question this worked for me:

>>> data = d[:, :, 0].astype(np.short)
>>> data[data==-32768] = data[data>0].min()
>>> data = data.astype(np.float32)

Upvotes: 2

Related Questions