CODEWITHSUNDEEP
pythonarraysnumpy
ELEC
ELEC

Reputation: 265

Fast way to transpose np.arrays and write them to file

I have situation where I'm writing .TDMS files to .txt files. TDMS file structure has groups that consist of channels that are in numpy array format. I'm trying to figure out the fastest way to loop over the groups and the channels and transpose them to "table" format. I have a working solution but I don't think it's thats fast.

Snippet of my code where file has been opened to tdms variable and groups variable is a list of groups in that .tdms file. Code loops over the groups and opens the list of channels in that group to channels variable. To get the numpy.array data from channel you use channels[list index].data. Then I just with column_stack add the channels to "table" format one by one and save the array with np.savetxt. Could there be a faster way to do this?

for i in range(len(groups)):
    output = filepath+"\\"+str(groups[i])+".txt" 
    channels = tdms.group_channels(groups[i]) #list of channels in i group
    data=channels[0].data #np.array for first index in channels list
    for i in range(1,len(channels)):
       data=np.column_stack((data,channels[i].data))       

    np.savetxt(output,data,delimiter="|",newline="\n")

Channel data is 1D array. length = 6200

channels[0].data = array([ 12.74204722,  12.74205311,  12.74205884, ...,
    12.78374288,  12.7837487 ,  13.78375434])

Upvotes: 1

Views: 1605

Answers (1)

hpaulj
hpaulj

Reputation: 231355

I think you can streamline the column_stack application with:

np.column_stack([chl.data for chl in channels])

I don't think it will save on time (much)

Is this what your data looks like?

In [138]: np.column_stack([[0,1,2],[10,11,12],[20,21,22]])
Out[138]: 
array([[ 0, 10, 20],
       [ 1, 11, 21],
       [ 2, 12, 22]])

savetxt iterates through the rows of data, performing a format and write on each.

Since each row of the output consists of a data point from each of the channels, I think you have to assemble a 2d array like this. And you have to iterate over the channels to do (assuming they are non-vectorizable objects).

And there doesn't appear to be any advantage to looping through the rows of data and doing your own line write. savetxt is relatively simple Python code.

With 1d arrays, all the same size, this construction is even simpler, and faster with the basic np.array:

np.array([[0,1,2],[10,11,12],[20,21,22]]).T

Upvotes: 1

Related Questions