CODEWITHSUNDEEP
pythonnumpyscipyinterpolation
Yorian
Yorian

Reputation: 2062

scipy.interpolate.interp2d: do I really have too many data points?

I have a set of elevation measurements that are on a X, Y grid. I'm trying to create a slice through the elevations (under an angle so not perfectly on the grid points). I thought of using the 2D interpolation method from scipy, but I get the error OverflowError: Too many data points to interpolate. I don't have an enormous array so I wonder why this is going wrong.

My data:

>>> XX.shape, YY.shape, depth_array.shape
((787, 1858), (787, 1858), (787, 1858))

>>> XX
array([[  0,   0,   0, ...,   0,   0,   0],
       [  1,   1,   1, ...,   1,   1,   1],
       [  2,   2,   2, ...,   2,   2,   2],
       ...,
       [784, 784, 784, ..., 784, 784, 784],
       [785, 785, 785, ..., 785, 785, 785],
       [786, 786, 786, ..., 786, 786, 786]])

>>> YY
array([[   0,    1,    2, ..., 1855, 1856, 1857],
    [   0,    1,    2, ..., 1855, 1856, 1857],
    [   0,    1,    2, ..., 1855, 1856, 1857],
    ...,
    [   0,    1,    2, ..., 1855, 1856, 1857],
    [   0,    1,    2, ..., 1855, 1856, 1857],
    [   0,    1,    2, ..., 1855, 1856, 1857]])

>>> depth_array
array([[0., 0., 0., ..., 0., 0., 0.],
    [0., 0., 0., ..., 0., 0., 0.],
    [0., 0., 0., ..., 0., 0., 0.],
    ...,
    [0., 0., 0., ..., 0., 0., 0.],
    [0., 0., 0., ..., 0., 0., 0.],
    [0., 0., 0., ..., 0., 0., 0.]])
# The depth array seems empty, but that's not the case (but that are quite a few zero values)

>>> interpolate.interp2d(YY, XX, depth_array, kind='linear')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/yorian/.pyenv/versions/3.7.5/envs/euromax/lib/python3.7/site-packages/scipy/interpolate/interpolate.py", line 229, in __init__
    self.tck = fitpack.bisplrep(x, y, z, kx=kx, ky=ky, s=0.0)
File "/Users/yorian/.pyenv/versions/3.7.5/envs/euromax/lib/python3.7/site-packages/scipy/interpolate/_fitpack_impl.py", line 956, in bisplrep
    msg=msg)
File "/Users/yorian/.pyenv/versions/3.7.5/envs/euromax/lib/python3.7/site-packages/scipy/interpolate/_fitpack_impl.py", line 48, in _int_overflow
    raise OverflowError(msg)
OverflowError: Too many data points to interpolate

I am now using RectBivariateSpline, but that seems to fit a spline, I want a 2D linear interpolation. Are (787, 1858) points really too many? If so, how can I implement this?

Upvotes: 1

Views: 1750

Answers (1)

tenhjo
tenhjo

Reputation: 4537

If you have a regular grid, it is sufficient to provide only 1D arrays for x and y coordinates. This is less computational expensive but I don't know if this is the reason for the error message in the case of the general grid.


import numpy as np
from scipy import interpolate

nx = 787
ny = 1858
depth_array = np.random.random((ny, nx))
res = interpolate.interp2d(range(nx), range(ny), depth_array, kind='linear')

I tried to reproduce your error and discovered roughly this behaviour when using the general grid x, y = np.meshgrid(np.arange(nx), np.arange(ny)):

  • nx*ny < 200000 : Works
  • nx*ny > 200000 : MemoryError
  • nx*ny > 250000 : OverflowError

Upvotes: 1

Related Questions