J Alex
J Alex

Reputation: 25

What is a memory efficient way to resize a large 3D array?

I have data from a sensor that addresses each event for each (x,y) coordinate pair for each given timestamp.

An example of the data from the sensor would be something like:

timestamp = [ 1, 1, 2, 3]
x = [ 25, 25, 60, 30]
y = [128, 120, 50, 50]

The issue I am running into is with large quantities of data coming in from the sensor. The resolution is 180x240 and I could expect up to 400,000 unique timestamps. My goal is to resize the data down to 28x28x400000. My current method in place is to do it in chunks, building part of the 3D array and performing the transformation to get it down to 28x28. A lot of the data is just zeros, so I have tried using sparse arrays but with no luck. Any suggestions on how I can improve this?

Upvotes: 0

Views: 61

Answers (0)

Related Questions