CODEWITHSUNDEEP
pythonarraysnumpymatrix
heresthebuzz
heresthebuzz

Reputation: 718

Incorrect reshaping of 3d array

I am doing some processing with a matrix of shape 1536 x 16 x 48. Here are some details about this dimensions:

So, i have 48 samples of 1536 values (representing 6 seconds of data) from the perspective of 16 different collectors.

My goal is to do some processing with this matrix, but to do so some transformations are needed first.

  1. Since 6 seconds is a large sequence, i want to split each of those samples into smaller sequences, specially 3 or 2 seconds. For instance, 1 sample of 6 seconds (1536) can be split into 2 samples of 3 seconds (768) or 3 samples of 2 seconds (512). The shape of this transformed matrix would go from 1536x16x48 to 768x16x96 (for 3 seconds) or 512x16x144 (for 2 seconds).

  2. Once i have this new matrix, i want to reshape it so i get one 2d matrix per observer and all values organized in columns instead of rows (e.g. for 2 seconds split: 512x16x144 => 144x512x16).

  3. Finally, i can now loop through 3rd dimension (16), do some computations (i.e. fast fourier transform) with each 2d matrix and reduce (sum) them all into a single one to get a final 144 x 512 matrix (in 2 seconds-split scenario).

The following code is what i made with numpy, but it is clearly wrong for me when i plot samples generated from this method.

def generate_fft_data(data,labels, n_seconds_split=3):
    x = 256 * n_seconds_split
    y = 16
    z = 48 * int(6/n_seconds_split)
    data = data.transpose(2,0,1).reshape(x,y,z).transpose(2,0,1)
    fft_data = []
    for electrode in range(data.shape[2]):
        y_t = fft(data[:,:,electrode])
        fft_data.append(np.abs(y_t))
    sum_of_ffts = np.add.reduce(fft_data) 
    return sum_of_ffts

I can provide more details if needed. Thanks in advance.

Upvotes: 1

Views: 166

Answers (2)

hpaulj
hpaulj

Reputation: 231395

In [88]: data = np.ones((1536,16,48))
In [89]:  x = 256 * 3
    ...:  y = 16
    ...:  z = 48 * int(6/3)
In [90]: x,y,z
Out[90]: (768, 16, 96)
In [91]: data.transpose(2,0,1).shape
Out[91]: (48, 1536, 16)
In [92]: data.transpose(2,0,1).reshape(x,y,z).shape
Out[92]: (768, 16, 96)

Reshaping [91] to [92] does not make sense. It works because the numbers 'add' up, but the distribution is messed up.

I think you want to first split the 1536 dimension into (2,768) (or the 3 equivalent), then move that 2 dimension to the end, and recombine it with the 48.

The details could vary, but here's one such sequence:

data1 = data.reshape(2,768,16,48).transpose(1,2,0,3).reshape(768,16,96)

Upvotes: 0

bb1
bb1

Reputation: 7863

You can try this:

def generate_fft_data(data,n_seconds_split=3):
    x_split = 6//n_seconds_split
    # split along 0 axis, stack pieces along the last axis and transpose
    arr = np.dstack(np.split(data, x_split)).transpose(2,0,1)
    fft_data = []
    for electrode in range(arr.shape[2]):
        y_t = fft(arr[:,:,electrode])
        fft_data.append(np.abs(y_t))
    sum_of_ffts = np.add.reduce(fft_data) 
    return sum_of_ffts

Upvotes: 1

Related Questions