CODEWITHSUNDEEP
pythonarraysnumpyappendnetcdf
svgspnr
svgspnr

Reputation: 57

How to concatenate data from multiple netCDF files with Python

I have some netCDF files, 24 for each of the directions (x, y, z) and 24 with values for different times. At the final point I have to plot the data for all time steps.

For the plotting I need to interpolate at specific point so I have to knew the nearest neighbor. My plan is to divide the data in to 3D cells so I don't have to search the nearest neighbor in the whole dataset.

So in my first step I read in my data files and create an array witch contains [x,y,z,v[:]] the coordinates of each point and the value for each time.

After that I calculate for each point the cell it belongs to and append it to a Array of 4 dimensions: x, y, z and v:

for vec in vecs:
    x_ind = int((vec[0]-xmin) / stepWidthX) 
    y_ind = int((vec[1]-ymin) / stepWidthY) 
    z_ind = int((vec[2]-zmin) / stepWidthZ) 

    if x_ind==gridPointsInXdirection:
        x_ind = x_ind-1
    if y_ind==gridPointsInYdirection:
        y_ind = y_ind-1
    if z_ind==gridPointsInZdirection:
        z_ind = z_ind-1
    #print z_ind, y_ind,x_ind

    XGridPoints[z_ind, y_ind, x_ind] = np.append(XGridPoints[z_ind, y_ind, x_ind], vec[0])
    YGridPoints[z_ind, y_ind, x_ind] = np.append(YGridPoints[z_ind, y_ind, x_ind], vec[1])
    ZGridPoints[z_ind, y_ind, x_ind] = np.append(ZGridPoints[z_ind, y_ind, x_ind], vec[2])
    VGridPoints[z_ind, y_ind, x_ind] = np.append(VGridPoints[z_ind, y_ind, x_ind], vec[3])

Where vecs is the array with all data points. So far it's working but my problem now is in VGridPoints: I have a long list of values and not a list of arrays. Is there a solution to append an array to an array element so that I can access it later something like:

x = XGridPoints[2,3,4][2]
y = YGridPoints[2,3,4][2]
z = ZGridPoints[2,3,4][2]
v[:] = VGridPoints[2,3,4][2]

When I take only one time step it's working but I have a large overdrive if I recalculate the cells and the nearest neighbour for each time step and they do not change the location over time.

Upvotes: 2

Views: 1519

Answers (1)

tiago
tiago

Reputation: 23502

Numpy is generally much more convenient if you know a priori the shape of the arrays you'll be working with. Things like appending to arrays suffer a performance penalty. I agree with Sebastian, that the easiest way (if possible) is to create an array large enough to hold everything (worst case scenario). If this is not possible then perhaps you can try with object arrays. For example, create an object array with the 3 spatial dimensions:

import numpy as N
XGridPoints = N.empty((nx, ny, nz), dtype='object')

(and the same for YGridPoints, ZGridPoints, VGridPoints.) And then you can set XGridPoints[z_ind, y_ind, x_ind] to a numpy array and append to that array as you need.

Upvotes: 2

Related Questions